There are two aspects: first that there is currently a much higher pace of critical vulnerabilities and exploits being discovered, and second that there is a much shorter window between discovery and large-scale exploitation.
People are saying the pace will settle down once LLMs find and fix all the bugs. Okay, sure, if you want to believe that sort of thing, but that still leaves the much shorter time window, and that's not going to go away anytime soon.
If you're on the side of the industry where your job or hobby involves breaking things, this is all very exciting. But if you're on the side of the industry where your job is to prevent things from being broken, then it really sucks.
I lost a full day yesterday because some big players discovered a thing, told other big players about it, "oopsed" enough public information to let everyone else know about it, and then left everyone else to figure out what to do with almost no information to act upon.
This cycle has happened several times this year now. If you're on the short list of big hosting providers, you get advance notice of what's coming and early fixes; if you're not, sucks to be you, hope you didn't have any plans today.
For people like me, this is just yet one more way LLMs are making my life materially worse.
The coordinated disclosure thing will have to stop soon. Ideally, if you found a vuln from the defensive side, you'd tell all the "good guys" about it and none of the bad guys. The world doesn't work that way, so instead, the compromise was to tell a small number of important big actors who were unlikely to leak it, so the most good could be done before everyone found out.
Now the simple knowledge that there might be an exploitable memory issue in some given piece of software is enough for bad actors to point LLMs at it and find the issue in hours. That will only keep getting worse.
At some point, it's going to become a speed game. If you find a vuln, you might as well disclose to everyone immediately. Anyone who runs internet-exposed software will need an agent watching for disclosures around the clock and triaging them automatically (either by upgrading, improving filtering with some protective shell, or taking the whole service offline until humans can help) to keep up with the offensive agents watching around the clock and attacking automatically.
I'm wondering how long till we see the first exploit that goes like this:
* A bad actor finds a real, theoretically severe but not easy-to-actually-exploit vulnerability in some popular software. E.g. it's an exploit that can get you pwned 100% with RCE, but only if you are using it in configuration X on platform Y with kernel version Z on a Tuesday.
* They design a patch that closes that vulnerability but subtly opens a more dangerous one known only to them. They privately test their patch against LLM reviewers, tuning the approach until the introduced vulnerability typically escapes detection by automated review.
* They "responsibly disclose" the original exploit and helpfully share the patch. In the hurry to fix the original exploit, the patch itself gets only cursory (claudy, codexy) review. By now this has happened so many times that just applying whatever code changes ClodSed were GTG is the status quo.
* When the new vulnerability is eventually discovered, they can plausibly say "oh my agent didn't catch that", which is true. In the meantime, they use the new exploit that only they know about.
One can defend by quarantining any submitted fix, developing your own without reading it. Then only after that compare them.
In any case you don't even need malice --- sincere reporter can vibe up a fix that accidentally introduced a vulnerability, owing to their own lack of domain expertise and model errors... and then this bad fix makes it in because in the thrall of confirmation bias you fail to notice its shortcomings. Reporter also presumably tested it against their own LLM, so that will pass too.
This happened recently in a cryptocurrency project and contributed to a significant exploit.
> If you find a vuln, you might as well disclose to everyone immediately.
This still doesn't work, since attackers are much more motivated and able to pay attention to such things.
If your goal is finding vulnerabilities / stealing money / whatever, you focus all your energy looking for new exploits.
The rest of the world has other things to do (listen to users, run services, whatever) and by definition cannot spend as much of their time watching every social media in the world for the chance of an announcement.
That's why anyone who has a lot to lose from a data breach had better either harden the shell of their system such that it's not vulnerable (very hard to do correctly) or automate the process of listening for and reacting to vulns.
> Now the simple knowledge that there might be an exploitable memory issue in some given piece of software is enough for bad actors to point LLMs at it and find the issue in hours.
It is all but certain there are exploitable memory issues in Chrome, Firefox, Node.js, Linux kernel, Nginx, etc.
Can we expect CVEs in a few hours? I'm sorry I ruined everyone's weekend.
You may joke, but I know someone who told Kimi to "go wild and fuzz filesystems in the Linux kernel", woke up the next morning to "26 bugs found and fixes prepared".
I understand several have been accepted since, and he stopped the fuzzers because his backlog of submissions to make was getting out of hand.
> People are saying the pace will settle down once LLMs find and fix all the bugs
Yeah I don't buy it either. Look at github right now, it is a total slopfest with people vibing projects they don't know the first thing about. Plenty of LLM use will be delivering code full of new exploits. It will be interesting to see the frequency of these exploits being found correlated to LLM friendly/unfriendly contribution policies once the dust has settled to tell for certain though.
I have the opposite view: these vibe-coded projects are MUCH higher security than even top-talent human.
Ask a top agent today to create a to-do app and watch it deploy enterprise grade security: anti CSRF, XSS, CSP, secure cookies, HMACs, rate-limiting of logins, and all kinds of things you never heard of.
I've asked it to implement a simple OAuth and saw it classify inbound IPs for private/public subnets, and put in workarounds for very theoretical DNS re-bindings attacks.
We all know LLMs love to over-engineer stuff, they are having a field day turning your static 3 page blog into crypto-exchange grade security.
It's right for small projects that are created by people who know how to code in geberal. But there are also two categories where this might not be true:
1. Simple utilities that a vlogger or a sales guy created to automate routine. Such utilities are created in a hurry, deployed with prodiding private key to an agents, and so on. No attention to security was payed.
2. New generation of enterprise-grade slop that is produced by a relatively large group of people that don't review their work properly due to insane TTM and amounts of code.
Yes, their code might be secure in terms of simple techniques (agents help a lot of course), but in such code bases it's extremely hard to find out what should be fixed, how to fix it without breaking something else, and how to prevent something similar from happening in the future.
> But if you're on the side of the industry where your job is to prevent things from being broken, then it really sucks
Is it? In using it at an opportunity to improve our devops pipelines, software supply chains, etc. Perhaps I'm weird and enjoy strange things? Not to mention that all these security incidents will require plenty of people to fix things = more paid work for us humans going forward.
> For people like me, this is just yet one more way LLMs are making my life materially worse.
I wonder if we start to hit a point where the pace that exploits are found and deployed increases faster than they get repaired, because there is a natural higher risk appetite for the bad actors, whereas the good actors still require careful thought and prudence. As a result of this, would the internet/digital economy largely seize up and crash? That would be bad, especially for a lot of people on this site. Maybe this leads to a regression in convenience, but requires more in-person interaction in order to function in society. You can already see this pattern today emerging in higher education. Maybe, in a deeper way, this improves humanity?
> People are saying the pace will settle down once LLMs find and fix all the bugs.
Only for projects that are in a maintenance state. New features will produce new bugs, unless everyone pays for the tokens to scan every release with the top-of-the-line agents or otherwise spends more time than many do ATM to ensure that changes and new code don't introduce significant exploit paths. Things will slow down once the low-hanging fruit from the past is mostly detected and pulled, but the level of vigilance will always have to be at least a little higher than it has been until recently.
Time between updates releasing and people installing them has been an increasing problem for some time, we are just going to have to be a lot more disciplined about that. This will be a problem for places that have strict availability requirements as they might have to compromise on the testing time they spend on new patches before updating their installations, and we might have to move to having more flexible maintenance windows (the old “an hour or two once a month after Patch Tuesday” hasn't cut it for some time and certainly won't going forward) to allow faster updates.
The bigger problem, compared to the patch-availability-to-general-use time which can be reduced somewhat via automation (though that might accelerate bad-patch based attacks…), is greater constraints for the time between bug discovery and patch release: as soon as there is a rumour of an issue many will start scanning for it (as that is easier now) so the time to wide exploitation is going to be much shorter. This might lead to buggy patches that add new issues of their own.
Perhaps we'll see a lot more projects having explicit LTS releases that only get security updates or signifiant bug fixes, which will reduce the attack surface area (and fault surface area) most users, letting those with less stability concerns deal with the bleeding edge.
> For people like me, this is just yet one more way LLMs are making my life materially worse.
> New features will produce new bugs, unless everyone pays for the tokens to scan every release with the top-of-the-line agents
Insufficient. Models available to you are already cyber-gimped today and seems likely that they will be moreso in the future (see the recent K3.1 announcement for example). There will be attackers that lack such handicap.
The web as we know it was never securable. LLMs made it obvious. We made a wrong turn somewhere back around JavaScript. Eventually we'll get fed up with the quick fixes and design something better. I don't blame the LLMs for this. I blame whoever made that wrong turn back in the 90's (and us, for not turning around sooner).
You're blaming JavaScript? The language that succeeded at delivering fully sandboxed rich client-side applications better than any other tech? With the worst incidents generally being 'just' various forms of fingerprinting?
Or, if you're thinking server-side JavaScript: would the web have been more secure running PHP and Python?
The language that made it normal to blindly run code that you found lying around on some page? Elsewhere we'd call that a remote code execution exploit. JavaScript just calls it another day at the office.
And it's not fully sandboxed. It betrays its user all the time by abusing unrestricted access to the network, or activating browser functionality that most of us would disable if we could.
I'm not saying it's THE bad decision, but it is the product of a generation of thinkers who were not thinking about threats the way we do now. If we were to go back in time and have the opportunity to do it differently, I think we would take that opportunity.
But we don't actually need a time machine. We can just accept some sunk costs and move forward anyhow.
Blindly running client side code (within a very carefully designed sandbox) is necessary for the modern web to operate, and people like the modern web.
We aren't going back to a page refresh every time you want to update something any time soon
Are you sure people like the modern web? It's increasingly used as a weapon against them. I don't know anybody under 20 who likes the modern web. As for the others, I think its more about nostalgia.
People will often claim to dislike something, even though they would dislike the alternative even more (and they might even be actively choosing against the alternative). Pay more attention to what people’s actions reveal than what they claim.
I guess the far closer(/better?) analogy would be something like people saying they dislike the ad-supported model for newspaper websites, YouTube, etc. However, many of those sites offer (or even heavily promote) options to get rid of the ads. (A number of the highest-regarded newspapers still even offer print versions too.)
The tech that made the old web still exists. Everything is a GET, a form POST, and needs a page reload. You can still use tables for layout like it's 1996.
1. I'm not proposing the users go back and live in the past. I'm proposing that we, the technologists, go back to the drawing board as it existed at that time and build a different future.
2. I'm not going to feel safe if I go make a web site that works that way. I'm going to feel safe if my bank does.
> I mean, Facebook, my bank, Amazon… these can all be created with no client side code. It would be a lot of refreshes, but it can be done.
HN could too - perhaps the biggest use on HN is not refreshing when up (or down) voting something. YouTube could do something similar - refreshing when 'liking' a video or getting more comments. But would you really find those to be a better experience for you? Even on actions that HN does refresh on, such as 'Next page', or submitting a comment, HN doesn't freeze sorts, so posts and/or comments you didn't see will often invisibly end up before the new location HN sent you to (or posts/comments you already saw earlier in the list now showing up "again" later in the list).
thaumaturgy 1 day ago
People are saying the pace will settle down once LLMs find and fix all the bugs. Okay, sure, if you want to believe that sort of thing, but that still leaves the much shorter time window, and that's not going to go away anytime soon.
If you're on the side of the industry where your job or hobby involves breaking things, this is all very exciting. But if you're on the side of the industry where your job is to prevent things from being broken, then it really sucks.
I lost a full day yesterday because some big players discovered a thing, told other big players about it, "oopsed" enough public information to let everyone else know about it, and then left everyone else to figure out what to do with almost no information to act upon.
This cycle has happened several times this year now. If you're on the short list of big hosting providers, you get advance notice of what's coming and early fixes; if you're not, sucks to be you, hope you didn't have any plans today.
For people like me, this is just yet one more way LLMs are making my life materially worse.
aftbit 1 day ago
Now the simple knowledge that there might be an exploitable memory issue in some given piece of software is enough for bad actors to point LLMs at it and find the issue in hours. That will only keep getting worse.
At some point, it's going to become a speed game. If you find a vuln, you might as well disclose to everyone immediately. Anyone who runs internet-exposed software will need an agent watching for disclosures around the clock and triaging them automatically (either by upgrading, improving filtering with some protective shell, or taking the whole service offline until humans can help) to keep up with the offensive agents watching around the clock and attacking automatically.
rspeele 1 day ago
* A bad actor finds a real, theoretically severe but not easy-to-actually-exploit vulnerability in some popular software. E.g. it's an exploit that can get you pwned 100% with RCE, but only if you are using it in configuration X on platform Y with kernel version Z on a Tuesday.
* They design a patch that closes that vulnerability but subtly opens a more dangerous one known only to them. They privately test their patch against LLM reviewers, tuning the approach until the introduced vulnerability typically escapes detection by automated review.
* They "responsibly disclose" the original exploit and helpfully share the patch. In the hurry to fix the original exploit, the patch itself gets only cursory (claudy, codexy) review. By now this has happened so many times that just applying whatever code changes ClodSed were GTG is the status quo.
* When the new vulnerability is eventually discovered, they can plausibly say "oh my agent didn't catch that", which is true. In the meantime, they use the new exploit that only they know about.
nullc 1 day ago
In any case you don't even need malice --- sincere reporter can vibe up a fix that accidentally introduced a vulnerability, owing to their own lack of domain expertise and model errors... and then this bad fix makes it in because in the thrall of confirmation bias you fail to notice its shortcomings. Reporter also presumably tested it against their own LLM, so that will pass too.
This happened recently in a cryptocurrency project and contributed to a significant exploit.
achierius 1 day ago
> the patch itself gets only cursory (claudy, codexy) review
FWIW Claude is actually quite good at discovering variants of existing bugs.
Arainach 1 day ago
This still doesn't work, since attackers are much more motivated and able to pay attention to such things.
If your goal is finding vulnerabilities / stealing money / whatever, you focus all your energy looking for new exploits.
The rest of the world has other things to do (listen to users, run services, whatever) and by definition cannot spend as much of their time watching every social media in the world for the chance of an announcement.
aftbit 1 day ago
thayne 1 day ago
1. This is also very hard to do correctly.
2. What automated steps would you take without any patches available to apply?
3. Potentially makes you more vulnerable to supply chain attacks
dns_snek 1 day ago
It is all but certain there are exploitable memory issues in Chrome, Firefox, Node.js, Linux kernel, Nginx, etc.
Can we expect CVEs in a few hours? I'm sorry I ruined everyone's weekend.
ben_w 23 hours ago
I understand several have been accepted since, and he stopped the fuzzers because his backlog of submissions to make was getting out of hand.
wasabinator 1 day ago
Yeah I don't buy it either. Look at github right now, it is a total slopfest with people vibing projects they don't know the first thing about. Plenty of LLM use will be delivering code full of new exploits. It will be interesting to see the frequency of these exploits being found correlated to LLM friendly/unfriendly contribution policies once the dust has settled to tell for certain though.
dist-epoch 1 day ago
Ask a top agent today to create a to-do app and watch it deploy enterprise grade security: anti CSRF, XSS, CSP, secure cookies, HMACs, rate-limiting of logins, and all kinds of things you never heard of.
I've asked it to implement a simple OAuth and saw it classify inbound IPs for private/public subnets, and put in workarounds for very theoretical DNS re-bindings attacks.
We all know LLMs love to over-engineer stuff, they are having a field day turning your static 3 page blog into crypto-exchange grade security.
alexjameson 20 hours ago
1. Simple utilities that a vlogger or a sales guy created to automate routine. Such utilities are created in a hurry, deployed with prodiding private key to an agents, and so on. No attention to security was payed.
2. New generation of enterprise-grade slop that is produced by a relatively large group of people that don't review their work properly due to insane TTM and amounts of code.
Yes, their code might be secure in terms of simple techniques (agents help a lot of course), but in such code bases it's extremely hard to find out what should be fixed, how to fix it without breaking something else, and how to prevent something similar from happening in the future.
linuxftw 1 day ago
Flere-Imsaho 1 day ago
Is it? In using it at an opportunity to improve our devops pipelines, software supply chains, etc. Perhaps I'm weird and enjoy strange things? Not to mention that all these security incidents will require plenty of people to fix things = more paid work for us humans going forward.
gWPVhyxPHqvk 1 day ago
I wonder if we start to hit a point where the pace that exploits are found and deployed increases faster than they get repaired, because there is a natural higher risk appetite for the bad actors, whereas the good actors still require careful thought and prudence. As a result of this, would the internet/digital economy largely seize up and crash? That would be bad, especially for a lot of people on this site. Maybe this leads to a regression in convenience, but requires more in-person interaction in order to function in society. You can already see this pattern today emerging in higher education. Maybe, in a deeper way, this improves humanity?
dspillett 1 day ago
Only for projects that are in a maintenance state. New features will produce new bugs, unless everyone pays for the tokens to scan every release with the top-of-the-line agents or otherwise spends more time than many do ATM to ensure that changes and new code don't introduce significant exploit paths. Things will slow down once the low-hanging fruit from the past is mostly detected and pulled, but the level of vigilance will always have to be at least a little higher than it has been until recently.
Time between updates releasing and people installing them has been an increasing problem for some time, we are just going to have to be a lot more disciplined about that. This will be a problem for places that have strict availability requirements as they might have to compromise on the testing time they spend on new patches before updating their installations, and we might have to move to having more flexible maintenance windows (the old “an hour or two once a month after Patch Tuesday” hasn't cut it for some time and certainly won't going forward) to allow faster updates.
The bigger problem, compared to the patch-availability-to-general-use time which can be reduced somewhat via automation (though that might accelerate bad-patch based attacks…), is greater constraints for the time between bug discovery and patch release: as soon as there is a rumour of an issue many will start scanning for it (as that is easier now) so the time to wide exploitation is going to be much shorter. This might lead to buggy patches that add new issues of their own.
Perhaps we'll see a lot more projects having explicit LTS releases that only get security updates or signifiant bug fixes, which will reduce the attack surface area (and fault surface area) most users, letting those with less stability concerns deal with the bleeding edge.
> For people like me, this is just yet one more way LLMs are making my life materially worse.
I'm with you there…
nullc 1 day ago
Insufficient. Models available to you are already cyber-gimped today and seems likely that they will be moreso in the future (see the recent K3.1 announcement for example). There will be attackers that lack such handicap.
verdverm 1 day ago
1. many have underinvested in security for a long time
2. many are checking out of the development process
I generally think we can improve software with Ai tools, but I'm skeptical the current incentives will lead to this.
__MatrixMan__ 1 day ago
vanviegen 1 day ago
Or, if you're thinking server-side JavaScript: would the web have been more secure running PHP and Python?
__MatrixMan__ 1 day ago
And it's not fully sandboxed. It betrays its user all the time by abusing unrestricted access to the network, or activating browser functionality that most of us would disable if we could.
I'm not saying it's THE bad decision, but it is the product of a generation of thinkers who were not thinking about threats the way we do now. If we were to go back in time and have the opportunity to do it differently, I think we would take that opportunity.
But we don't actually need a time machine. We can just accept some sunk costs and move forward anyhow.
saltcured 1 day ago
And before that, whatever Microsoft "macro" or embedded code variant was popular that week in all those document formats...
ifwinterco 1 day ago
We aren't going back to a page refresh every time you want to update something any time soon
__MatrixMan__ 1 day ago
no-name-here 1 day ago
People will often claim to dislike something, even though they would dislike the alternative even more (and they might even be actively choosing against the alternative). Pay more attention to what people’s actions reveal than what they claim.
(This happens not just in tech.)
pwdisswordfishq 1 day ago
no-name-here 21 hours ago
cowlevel 17 hours ago
olyjohn 10 hours ago
icedchai 1 day ago
__MatrixMan__ 1 day ago
2. I'm not going to feel safe if I go make a web site that works that way. I'm going to feel safe if my bank does.
pwdisswordfishq 1 day ago
BoxwoodSeed 1 day ago
Give me a https://motherfuckingwebsite.com/ instead.
JKCalhoun 1 day ago
vanviegen 1 day ago
preg_match 1 day ago
But 99% of web applications are forms and displaying stuff. All JS accomplished is making it look prettier and enhanced tracking.
I mean, Facebook, my bank, Amazon… these can all be created with no client side code. It would be a lot of refreshes, but it can be done.
no-name-here 1 day ago
HN could too - perhaps the biggest use on HN is not refreshing when up (or down) voting something. YouTube could do something similar - refreshing when 'liking' a video or getting more comments. But would you really find those to be a better experience for you? Even on actions that HN does refresh on, such as 'Next page', or submitting a comment, HN doesn't freeze sorts, so posts and/or comments you didn't see will often invisibly end up before the new location HN sent you to (or posts/comments you already saw earlier in the list now showing up "again" later in the list).
lloeki 1 day ago
Not saying that's exactly what should be done, but an example that there may be another way than full-on generic code execution wrapping XHR.