Today in InfoSec Job Security News:

I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.

So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.

github.com/search?q=author%3Ac…

As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.

This entry was edited (17 hours ago)
in reply to da_667

what's funny to me, is that there were influencers on linkedin a few days ago claiming claudecode could find vulnerabilities in code faster than humans, and they're like "look at all these openssl vulns it found!" now I'm like. "well no shit its finding vulnerabilities, when its the one introducing them."
in reply to da_667

@da_667 I demoed that very thing recently. Prompted up a form page and visually I could see a handful of basic JavaScript issues.

Ask Claude to review the code it generated for vulns using OWASP Top 10. And it finds them.

That’s just bonkers. Sure, a lazy initial prompt so it’s all my fault, really.

@GossiTheDog

in reply to Kevin Beaumont

I wonder across the industry how common is it for orgs to skip static code analysis, or other code vulnerability scans as part of their pipelines? Even then how many of those scans are actually effective?

Looks like AI is potentially an insider threat, and code generated by it has to be treated accordingly, even in the case of it being generated by project members and "reviewed"

in reply to Kevin Beaumont

I'm anti-AI. I used program generators long ago - they didn't work. They aren't maintainable. Major updates required complete rewrites.

Now there's AI. It's a manager's wet dream...until it isn't.

...but look how productive AI is. It can whip out code as fast as a gossip can spread noise. Sure, there will be glitches, but they'll be fixed when found.

What about the $$$$$ liability of glitches that are not found?

in reply to C64Whiz

in reply to ndevenish

morry040 reshared this.

in reply to Cassandrich

in reply to Kevin Beaumont

Reminds me of this paper from a year ago.

arxiv.org/abs/2502.17424

LLM trained (fine tuned) on code with security vulns, but not told it was vulnerable code, not only reproduced vulnerable code (expected) but also showed spontaneous ethical misalignment "judgment" in other domains.

It's a really interesting read.

If the model is producing OWASP top 10 errors like directory traversal, would seem likely it was trained on vulnerable code.

Hmmm.