Please regulate us, now! If we don’t get regulatory capture ASAP I’m toast. I mean look man I just had to go beg for tens of billions more cash so I can keep the lights on and my IPO is looking more and more screwed every day. Please, I’m begging you. What more do I need to say to convince you?
I'm fascinated by this forum's take on AI regulations.
Whenever there is an adverse event, people say of course it should be regulated. The EU AI Act is a good start but we need more. These labs need to be held accountable just like all industries that can impact others. Regulate the chips, regulate the models, regulate the companies.
When these companies ask for regulations, people say they are trying to do regulatory capture. They are trying to ban opensource models, and cement their place as the AI arbiters and ban all others. We need competition and open weight models and China is leading the charge.
I understand that the 2 position aren't exactly opposed. We probably need regulations, and ideally ones that don't produce regulatory capture. That is extremely hard though. Almost all regulations end up creating a form of regulatory capture. Any regulation compliance has a cost. The more complex and wide reaching the regulations are, the more expensive it's to comply and the more they cement the existing players and make it harder for new ones to enter the market. FAA regulations makes it so we don't have dozens of passenger airplane startups that are trying crazy wacky things. Food services are regulated too, but the compliance is fairly doable by mom and pop stores, yet probably too high for your unlicensed pop-up taco stand.
I haven't really seen any meaningful discussion about what AI regulation should be. Only extremes of "No regulations, all is fair" or "no one should have access to running or developing models but OpenAI and Anthropic".
The problem with regulation is that it won't (can't) solve the problem. You'd have to go totally draconian to stop law breaking people within the US from doing matrix multiplies, and then you'd have to disconnect the US internet from all the countries who won't give a crap about US regulation.
So yeah, US regulation is ONLY about capturing US consumers. This isn't food or airplanes.
Further, its about ensuring the open weights models never catchup/exist so these guys dont turn into a hardware reit like they should and they can keep their insane margins
Regulate the industries to prevent anticompetitive practices, and hold companies and individuals responsible for the consequences of their actions. I honestly don't see the need for many, if any, AI-specific regulations if the laws we already have are consistently enforced and AI "agency" is not accepted as a means to limit liability.
> hold companies and individuals responsible for the consequences of their actions.
Are you referring to the model providers or the companies/individuals using the models or both?
Like I totally get it about the one instructing the model that result in some harm (be it the model provider themselves or ones using it). If a person buys a gun and commits a crime, they are held responsible not the gun maker or seller (assuming the seller followed selling regulations too, and it wasn't the gun maker or the gun seller who committed the crime obviously). That's already the case though. For example, Cyber attacks are a crime regardless how you do it. It's not like a company can trade publicly today that's using AI to attack other companies intentionally. I'm sure if huggingface or the government wanted to sue OpenAI for cyber attacks, they could be successful in getting paid for damages or getting a fine imposed. All parties are just choosing not to at the moment, but if a company makes a habit of doing that, they will get sued. If Chase or Amazon tell Meta to stop Muse from using them, Meta will have to abide or they will get sued.
On the other hand, holding model developers and providers responsible for the model's results would be a form of AI regulation, right? Like you can regulate social media websites by holding them responsible for their user's posts too or deregulate them by not doing that, we ended up with the latter. If Section 230 never happened in the 1990s, then the internet would look very very different today.
Furthermore, it seems to me that holding model developers and providers responsible for the results of their models would be that regulatory capture every one is against, no? like how can an opensource model developer exist without getting sued into oblivion. And how can a small model provider survive if there is always a looming disaster they might be responsible for. As a result only the big players would exist and they would heavily control who gets access to these models through a lot of inner deals and contracts on how they can be used.
Are you conflating regulation and accountability? I haven't seen anyone (without a vested interest) ask for the former as a response to an adverse event, only the latter. There is a difference.
Regulation should require model auditing, which should be fully automated, free(or extremely affordable), and time-bound, irrespective of who the client is.
What Altman, Dario and other such folks are trying to do is put a price on getting the model audited, by playing on these: price, timelines, red-tape.
Maybe a test-suite, kept continuously updated, similar to those benchmarks we already have for measuring the intelligence of these models. Like arc-agi, or terminal bench, or more.
You seem to think that regulation means hitting the brakes. We're way past that point. This is "we need to mobilize an AT squad" territory.
Sure, research will continue in secret, just like nuclear, biological and chemical weapons. It's just that there's going to be an economy backing it up, with MAD ensuring the attacks are "civil", as Axl once said.
Yes, I'm. I didn't mean for my list of extremes to be an exhaustive list. But yes, NOBODY (in a "butlerian jihad" sense) is just another example in the list of extreme takes. Only governments or military is another extreme.
Some of us want to nationalize these companies. What we’re not going to allow is landed gentry to profit for generations from stealing all the content they could scrape.
Yes and no. These things have always existed and were typically known but not broadly exploited. A huge part of what nation state hackers do is exploit these things for intelligence use.
So I would agree that this is bringing more of this to light, but the idea that unknown exploitable bugs can get weaponized being a new thing is naive.
If anything putting all this out in the open makes things more secure, not less, in the long run as it’s harder for hackers to know and exploit them before they’re known as they don’t survive undetected for long. So a bit painful in the short term but we get more secure infra in the long term.
Still not a reason to give Sam regulatory capture as an out to saving OpenAI.
I don't think this will mean holding back, as nothing will hold back whatever is providing tech to the myriad of parties interested in poking holes. Instead this points to common defense, as anyone who doesn't have access to defensive tech will be toast.
I'm so much less worried about AI regulation than I am about vulnerability research legislation, which is much easier to half-ass and apply without goring any major economic oxes.
I genuinely would like to know your opinion on what the next steps would be then? If every single exploit was available tomorrow accompanied by a working exploit to operationalize it, or combine it in a chain to compromise any software that has such a vulnerability, would you post it on the internet?
I know you a bit by reputation so that was my assumption. I assume you mean responsible disclosure? I thought your statement was implying otherwise, maybe the unsaid implication was against the "extortion" argument?
In any case, what I want to say is this: All technology has tradeoffs but this has shown we are locked in a game theory mechanic not dissimilar to extortion AND the security implications are part of that.
- We have to build AI or someone else will.
- You need our tools to protect yourself from our tools or similar ones
- You need to use AI or someone using AI will take your job.
Many examples but it is very "i created the disease to sell you the cure". If this were any other technology, we would expect more reasonable mitigations, extraneous factors like software bugs existing aside. It speaks to a fundamental lack of control on the part of the labs that we take for granted: they simply don't have control of the PRODUCT, much less how it is even used.
Absolutely not. "Responsible disclosure" is a coercive Orwellian term coined by vendors, most importantly Microsoft, in the early aughts in order to set a norm that vulnerability researchers should work on vendor timelines and with vendor disclosure constraints.
Finding vulnerabilities that already exist in code can't be "creating the disease", for obvious reasons.
Ok. I was sympathetic to that historically for all the reasons you stated. If you put the effort in, why are you constrained. I think we are in a different world now. So what would you do in my thought experiment?
You are the QA head and the new model could literally find and exploit ANY bug? Would you release the model? Genuinely curious. Its not totally science fiction. I think there is a difference in a couple smart people punishing 1 sloppy company that underinvested in security. The internet is a stack of protocols, hardware, infrastructure, applications etc. It could be the case that almost everyone has capabilities on par with yours. Great you say, security will improve! However, there will be a meltdown in the middle where potentially digital currency is exploited, exchanges, banks, utilities. It isn't just a team working for 3 months, finding some issues in a company.
I cannot help to think that the idea of go fast and break things had something to do with the current state of affairs. I would say that the next step should be slow and consolidate, but it seems like we are not heading in that direction.
At this point I don't know what will be worse for the global economy the AI Bubble bursting, or the gigantic crash all this vulns might bring if they force the stop because we went too far and everything came crumbling down...
Ah yes. Everyone eventually dies, we just shorthen the wait.
This is like telling that bank vaults always were breachable, and giving out explosives to everyone just made them get breached sooner.
(edit: fact of existance of a vulnerability is not a binary yes/no. the amount of effort needed to exploit even known vulnerabilities is usually staggering, requiring months from very capable individuals with very specific skillset.)
In your analogy, one phenomenon concerns a living organism and the process of senescence, and the other a piece of information. I don't see how you connect the two.
Both my analogies use hyperbola to comically exaggerate the claimed lack of any difference. just as claiming that advancing a demise of a human makes no difference because it will eventually happen, or claiming that breaching a bank in practice is not a big deal because it was possible in theory, so also claiming that theoretic possibility of a vulnerability being exploited is more or less the same as many of them being exploited at the same time.
The covid was not a lot deadlier and not a lot more contagious than normal seasonal flu. That "not a lot" overloaded hospitals and caused global shutdown.
The increase in rate at which software is being breached right now in in no way "not a lot". Devs were overloaded even without this deluge. We can tank one such php exploit. Then another one. But for how long? If all your smartcards, secure elements, cloud providers, wireless protocols, drivers, government registries etc etc will be having weekly breaches of this magnitude, won't the effect be systemic?
None of these analogies are apposite. They don't respond to the observation I made: the vulnerabilities we're discussing exist regardless of what OpenAI does.
The fact that vulnerabilities exist is not being disputed, and is not causing any controversy. The fact that vulnerabilities are getting exploited does. Even having AI that would simply find all existing vulnerabilities and list them would not be as destructive as current situation. The same AI that finds the previously unknown vulnerability also finds the way to exploit it (NOT THE SAME THING), writes actually usable exploit (also not the same thing), then orchestrates and oversees the mass campaign. THAT is what makes current situation unsustainable.
Let's all recall old times when vulnerabilities of this magnitude were discovered - windows defender had some nice ones, linux kernel did, a lot of software did.
They got discovered, communicated to vendor, patches were made, people updated.
Now? You have HOURS from discovery to full-out campaign. You published a patch? HOURS before it is disassembled, all fixed bugs PoC-ed and full out campaign started. You cannot update anything meaningfully in that window of time.
Your premise, whether you mean it to be or not, is that if OpenAI wasn't working with researchers to publish vulnerabilities, they wouldn't be exploited. That's false.
(where in the article did you read about OpenAI working with researchers to publish vulnerabilities?)
Yes, exactly, that is my premise (minus the "working with researchers" bit). The current rate of exploitation is direct consequence of actions of Anthropic, OpenAI, Alibaba and whoever else is making these things. How can I know my false premise is true? By looking at the situation just a year ago, that is how!
You are focused on the binary: vulnerability exists or it does not. It can be exploited or it can not.
cmiles8 15 hours ago
Please regulate us, now! If we don’t get regulatory capture ASAP I’m toast. I mean look man I just had to go beg for tens of billions more cash so I can keep the lights on and my IPO is looking more and more screwed every day. Please, I’m begging you. What more do I need to say to convince you?
eddythompson80 15 hours ago
Whenever there is an adverse event, people say of course it should be regulated. The EU AI Act is a good start but we need more. These labs need to be held accountable just like all industries that can impact others. Regulate the chips, regulate the models, regulate the companies.
When these companies ask for regulations, people say they are trying to do regulatory capture. They are trying to ban opensource models, and cement their place as the AI arbiters and ban all others. We need competition and open weight models and China is leading the charge.
I understand that the 2 position aren't exactly opposed. We probably need regulations, and ideally ones that don't produce regulatory capture. That is extremely hard though. Almost all regulations end up creating a form of regulatory capture. Any regulation compliance has a cost. The more complex and wide reaching the regulations are, the more expensive it's to comply and the more they cement the existing players and make it harder for new ones to enter the market. FAA regulations makes it so we don't have dozens of passenger airplane startups that are trying crazy wacky things. Food services are regulated too, but the compliance is fairly doable by mom and pop stores, yet probably too high for your unlicensed pop-up taco stand.
I haven't really seen any meaningful discussion about what AI regulation should be. Only extremes of "No regulations, all is fair" or "no one should have access to running or developing models but OpenAI and Anthropic".
xscott 14 hours ago
So yeah, US regulation is ONLY about capturing US consumers. This isn't food or airplanes.
snapplebobapple 13 hours ago
jasomill 5 hours ago
eddythompson80 4 hours ago
Are you referring to the model providers or the companies/individuals using the models or both?
Like I totally get it about the one instructing the model that result in some harm (be it the model provider themselves or ones using it). If a person buys a gun and commits a crime, they are held responsible not the gun maker or seller (assuming the seller followed selling regulations too, and it wasn't the gun maker or the gun seller who committed the crime obviously). That's already the case though. For example, Cyber attacks are a crime regardless how you do it. It's not like a company can trade publicly today that's using AI to attack other companies intentionally. I'm sure if huggingface or the government wanted to sue OpenAI for cyber attacks, they could be successful in getting paid for damages or getting a fine imposed. All parties are just choosing not to at the moment, but if a company makes a habit of doing that, they will get sued. If Chase or Amazon tell Meta to stop Muse from using them, Meta will have to abide or they will get sued.
On the other hand, holding model developers and providers responsible for the model's results would be a form of AI regulation, right? Like you can regulate social media websites by holding them responsible for their user's posts too or deregulate them by not doing that, we ended up with the latter. If Section 230 never happened in the 1990s, then the internet would look very very different today.
Furthermore, it seems to me that holding model developers and providers responsible for the results of their models would be that regulatory capture every one is against, no? like how can an opensource model developer exist without getting sued into oblivion. And how can a small model provider survive if there is always a looming disaster they might be responsible for. As a result only the big players would exist and they would heavily control who gets access to these models through a lot of inner deals and contracts on how they can be used.
argee 14 hours ago
freakynit 15 hours ago
What Altman, Dario and other such folks are trying to do is put a price on getting the model audited, by playing on these: price, timelines, red-tape.
Maybe a test-suite, kept continuously updated, similar to those benchmarks we already have for measuring the intelligence of these models. Like arc-agi, or terminal bench, or more.
za_creature 15 hours ago
Sure, research will continue in secret, just like nuclear, biological and chemical weapons. It's just that there's going to be an economy backing it up, with MAD ensuring the attacks are "civil", as Axl once said.
eddythompson80 14 hours ago
I’m explicitly criticizing that position.
za_creature 14 hours ago
Seems like you're trying to move the discussion towards the tradeoffs of more vs less regulation, both of which are "moderate" talking points.
Your "extremes" are "free for all" and "nobody but big tech"
Here's a true extreme: NOBODY
eddythompson80 10 hours ago
za_creature 7 hours ago
You only mentioned one of them, and claimed that the current status quo is the other. I find that disingenuous.
edit: current status quo is "best models are only for internal use", broadly equivalent to "nobody except openai and antrophic"
eddythompson80 4 hours ago
- No regulations what so ever, it's free for all
- Complete ban on all AI
- Only OpenAI and Anthropic can do AI
- Only the Government, the Military, NASA or other entities approved explicitly by Congress can do AI.
That's not the current status quo either
petcat 15 hours ago
But the reality is that right now it's an arms race and the only countries crying for regulations are the ones that know they've already lost.
nozzlegear 5 hours ago
analognoise 14 hours ago
ComplexSystems 15 hours ago
cmiles8 15 hours ago
So I would agree that this is bringing more of this to light, but the idea that unknown exploitable bugs can get weaponized being a new thing is naive.
If anything putting all this out in the open makes things more secure, not less, in the long run as it’s harder for hackers to know and exploit them before they’re known as they don’t survive undetected for long. So a bit painful in the short term but we get more secure infra in the long term.
Still not a reason to give Sam regulatory capture as an out to saving OpenAI.
ComplexSystems 15 hours ago
za_creature 15 hours ago
Nobody of course, this is America!
siva7 15 hours ago
threatofrain 12 hours ago
tptacek 10 hours ago
pmlnr 16 hours ago
This is the textbook definition of racketeering.
tptacek 15 hours ago
wand3r 15 hours ago
tptacek 15 hours ago
wand3r 14 hours ago
In any case, what I want to say is this: All technology has tradeoffs but this has shown we are locked in a game theory mechanic not dissimilar to extortion AND the security implications are part of that.
- We have to build AI or someone else will.
- You need our tools to protect yourself from our tools or similar ones
- You need to use AI or someone using AI will take your job.
Many examples but it is very "i created the disease to sell you the cure". If this were any other technology, we would expect more reasonable mitigations, extraneous factors like software bugs existing aside. It speaks to a fundamental lack of control on the part of the labs that we take for granted: they simply don't have control of the PRODUCT, much less how it is even used.
tptacek 14 hours ago
Finding vulnerabilities that already exist in code can't be "creating the disease", for obvious reasons.
wand3r 14 hours ago
You are the QA head and the new model could literally find and exploit ANY bug? Would you release the model? Genuinely curious. Its not totally science fiction. I think there is a difference in a couple smart people punishing 1 sloppy company that underinvested in security. The internet is a stack of protocols, hardware, infrastructure, applications etc. It could be the case that almost everyone has capabilities on par with yours. Great you say, security will improve! However, there will be a meltdown in the middle where potentially digital currency is exploited, exchanges, banks, utilities. It isn't just a team working for 3 months, finding some issues in a company.
Or is this way overblown?
tptacek 13 hours ago
ericst 14 hours ago
At this point I don't know what will be worse for the global economy the AI Bubble bursting, or the gigantic crash all this vulns might bring if they force the stop because we went too far and everything came crumbling down...
112233 15 hours ago
This is like telling that bank vaults always were breachable, and giving out explosives to everyone just made them get breached sooner.
(edit: fact of existance of a vulnerability is not a binary yes/no. the amount of effort needed to exploit even known vulnerabilities is usually staggering, requiring months from very capable individuals with very specific skillset.)
tptacek 15 hours ago
112233 15 hours ago
The covid was not a lot deadlier and not a lot more contagious than normal seasonal flu. That "not a lot" overloaded hospitals and caused global shutdown.
The increase in rate at which software is being breached right now in in no way "not a lot". Devs were overloaded even without this deluge. We can tank one such php exploit. Then another one. But for how long? If all your smartcards, secure elements, cloud providers, wireless protocols, drivers, government registries etc etc will be having weekly breaches of this magnitude, won't the effect be systemic?
tptacek 15 hours ago
112233 15 hours ago
Let's all recall old times when vulnerabilities of this magnitude were discovered - windows defender had some nice ones, linux kernel did, a lot of software did. They got discovered, communicated to vendor, patches were made, people updated.
Now? You have HOURS from discovery to full-out campaign. You published a patch? HOURS before it is disassembled, all fixed bugs PoC-ed and full out campaign started. You cannot update anything meaningfully in that window of time.
tptacek 15 hours ago
112233 14 hours ago
Yes, exactly, that is my premise (minus the "working with researchers" bit). The current rate of exploitation is direct consequence of actions of Anthropic, OpenAI, Alibaba and whoever else is making these things. How can I know my false premise is true? By looking at the situation just a year ago, that is how!
You are focused on the binary: vulnerability exists or it does not. It can be exploited or it can not.
That is not what is causing the issues!