Post by @trilobite@lemmy.ml

2026-10-11 15:46 UTC

Have we been running insecure software since the arrival of AI? I do about a monthly update of my Debian machines. Today I did the usual monthly update and noticed loads of packages uploading from the “security” branch. I’ve had the impression (just an impression) that more and more packages are being updated from security over the past many months. Trixie has been out for a while now and I would have thought that less and less updates should be coming from security. Has anyone had this sensation and if so has anyone linked this to AI finding more holes in OSS that would have normally been found by human scrutiny? Has there been any official research into this? And if this is true, does it mean that we have been running insecure software all along? Or is there is there a risk that AI start filing patches that actually make it less secure because software maintainers are not being rigorous in verifying who is submitting the patch? AI must be changing the paradigm of software development more than any other area of our lives and I can’t still work out if in a better way or not.

Replies (5)

@slazer2au@lemmy.world 2026-10-11 16:07

We have been running insecure software since software was invented. AI has just accelerated finding what may be bugs. Wanacry/eternal blue existed before AI, Confliker existed before AI, SQLslamer, heartbleed, log4j all existed before AI.

Open ##4988820
@TrollAccount69@lemmy.ml 2026-10-11 17:00

There are two factors at play here: Ai is fast at finding and patching bugs. It’s just true. A lot of the bugs aren’t super high priority but can be daisy chained along with another and another and another exploit to eventually have serious consequences. Ai makes it much easier to recognize components of an exploit chain. What would just be a temporary hang for a tester or qa person is immediately recognized as a oob read, deserialization error or any number of other little bitty things that can make up one component of an exploit chain. So yes, we have been using insecure software this whole time.

Open ##4989408
@ISO@lemmy.zip 2026-10-11 19:51

Let me do the humble action of quoting myself 🙂 from 5 months ago: There has been murmurings, mainly from non-technical people, about how “AI” will render advancements in safer type systems nearly useless, because the magic ^(mushroom)^ AI will just find all the issues in code written in older languages. What they don’t realize is that the effect will be reversed. Many established projects that come with a high reputation, and a veneer of maturity, indestructibility, and meticulousness, will simply, and perhaps unfairly, lose that perception under the continuous barrage of potentially high impact bugs and vulnerabilities surfaced by these tools, with not enough human bandwidth to keep up with them, and with new code susceptible to the same problems repeating over and over. This will effectively lead to an even harder push for adopting technologies that prevent a good chunk of these bugs from ever happening at any point, not the other way around.

Open ##4990782
@Hal@piefed.nz 2026-10-12 00:50

I don’t think anyone has mentioned this yet, but the idea of AI hacking tools has also led to a lot more human effort in finding and fixing security issues. A while back there was an article about AI security tools and something about over 200 security vulnerabilities patched (perhaps in Chrome?). But in the detail of the article, it mentioned only 4 were found by AI. The rest were found by humans trying to get there before people with AI tools found them.

Open ##4993697
@technocrit@lemmy.dbzer0.com 2026-10-12 14:43

Has anyone had this sensation and if so has anyone linked this to AI finding more holes in OSS that would have normally been found by human scrutiny? A major point of automated security checks is that humans didn’t find these vulnerabilities and might never have found them without computational aids.

Open ##5003855