Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
github.com/search?q=author%3Ac…
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
This entry was edited (18 hours ago)
like this
reshared this
Being Left Behind Enjoyer
in reply to Kevin Beaumont • • •Fennix
in reply to Being Left Behind Enjoyer • • •I mean, if climate change becomes fixed eventually there won't be any more cancer, so they aren't completely wrong.
Pier Hegeman
in reply to Fennix • • •cR0w
in reply to Kevin Beaumont • • •Alan Langford 🇨🇦🧤🧊摏
in reply to cR0w • • •Fennix
in reply to cR0w • • •@cR0w @GossiTheDog
Klaus Frank
in reply to cR0w • • •@cR0w
Well I guess this must be what they meant by the saying "only idiots don't learn anything from their failures and smart people even learn from the failures of others, not just their own."
#quote
da_667
in reply to Kevin Beaumont • • •@GossiTheDog
da_667
in reply to da_667 • • •your auntifa liza 🇵🇷 🦛 🦦 reshared this.
B'ad Samurai 🐐🇺🇦
in reply to da_667 • • •@da_667 I demoed that very thing recently. Prompted up a form page and visually I could see a handful of basic JavaScript issues.
Ask Claude to review the code it generated for vulns using OWASP Top 10. And it finds them.
That’s just bonkers. Sure, a lazy initial prompt so it’s all my fault, really.
@GossiTheDog
Ron Bowes
in reply to B'ad Samurai 🐐🇺🇦 • • •Brian
in reply to da_667 • • •kwayk42
in reply to da_667 • • •@da_667 @GossiTheDog
Brian
in reply to da_667 • • •crazyeddie
in reply to Brian • • •@Drat @da_667 It does. In the form I was really fond of it's $50 per 750 ml and makes you say stupid shit like, "GASP...that's really smooth...." and then shove your head up your ass.
But I'm actually sick to death of that kind of oblivion. The shit I have to unsee just keeps adding up as does the shame of letting shit pass by unopposed.
Ben Royce 🇺🇦 🇸🇩
in reply to crazyeddie • • •@crazyeddie @Drat @da_667
strangely poetic
and absolutely true
Bradley
in reply to da_667 • • •@da_667 @GossiTheDog
tuban_muzuru
in reply to da_667 • • •@da_667 @GossiTheDog
HighlandLawyer
in reply to da_667 • • •@da_667 @GossiTheDog
DJGummikuh
in reply to Kevin Beaumont • • •Aleksandr Koltsoff
in reply to DJGummikuh • • •your auntifa liza 🇵🇷 🦛 🦦 reshared this.
Martin Seeger
in reply to Kevin Beaumont • • •h0h0kam
in reply to Kevin Beaumont • • •Androcat
in reply to h0h0kam • • •Sensitive content
The LLM can fuck up your project much faster than human developers ever could.
"Musty Bits" McGee
in reply to Kevin Beaumont • • •Simon Zerafa (Status: 😊)
in reply to Kevin Beaumont • • •Harry Sintonen
in reply to Kevin Beaumont • • •It's almost as if the language models are actually not intelligent at all.
Who would have thought!?
Bastian
in reply to Kevin Beaumont • • •@GossiTheDog
Tero Hänninen
in reply to Kevin Beaumont • • •I like the part where people are using Claude to write CLAUDE.md to explain Claude about directory traversal.
Nothing in this supply chain could ever go wrong.
Sebastian Bergmann
in reply to Kevin Beaumont • • •Brian David
in reply to Kevin Beaumont • • •The Penguin of Evil
in reply to Kevin Beaumont • • •Richard Hughes
in reply to Kevin Beaumont • • •Anthony
in reply to Kevin Beaumont • • •1705679109.757852
buc.ciDaniel Lakeland
in reply to Kevin Beaumont • • •The real question is why does a bot have commit privileges on a "major web framework"?
i mean the answer is probably because google owns the repo probably... but why?
Alun Jones
in reply to Kevin Beaumont • • •spinnyspinlock
in reply to Kevin Beaumont • • •Keith Lawson
in reply to Kevin Beaumont • • •This was literally the first major security mistake I made in my early days as a Perl developer and I don't imagine it's that uncommon. Claude has probably been trained with a truckload of code with these vulnerabilities.
That's okay because we run everything in single-purpose Docker containers now though, right? /s
Steve Hersey
in reply to Keith Lawson • • •I keep pointing out to my coworkers that these clankers are trained on StackOverflow posts that contain code examples followed by "here's what I wrote, why doesn't it work?"
Charlie Stross
in reply to Steve Hersey • • •Rachel
in reply to Kevin Beaumont • • •I wonder across the industry how common is it for orgs to skip static code analysis, or other code vulnerability scans as part of their pipelines? Even then how many of those scans are actually effective?
Looks like AI is potentially an insider threat, and code generated by it has to be treated accordingly, even in the case of it being generated by project members and "reviewed"
spinnyspinlock
in reply to Kevin Beaumont • • •Eric Likness
in reply to Kevin Beaumont • • •That Claude is a "clod", and boy does Claude get around I tell ya'. 🏃
Claude is everywhere you want an exploit to be. 🚨
claudex
in reply to Kevin Beaumont • • •kpcyrd 🏴
in reply to Kevin Beaumont • • •Klaus Frank
in reply to Kevin Beaumont • • •can you please post this also over on LinkedIn for all of the corporate people and CEOs to see?
We can't highlight how much of a liability generator all of this is...
Kyle
in reply to Kevin Beaumont • • •pinkforest(she/her) 🦀
in reply to Kevin Beaumont • • •Victor Nava
in reply to Kevin Beaumont • • •Fritz Adalis
in reply to Kevin Beaumont • • •Fritz Adalis
in reply to Fritz Adalis • • •seism0saurus
in reply to Kevin Beaumont • • •funnymonkey
in reply to Kevin Beaumont • • •OMFG.
Eric Likness
in reply to funnymonkey • • •@funnymonkey We don't need Skyntr becoming sentient to trigger the End o' Days.
We got Claude, happily vibing/making 2.1M commits while we were asleep.😴
Sassinake! - ⊃∪∩⪽
in reply to Kevin Beaumont • • •Dave
in reply to Kevin Beaumont • • •I'm anti-AI. I used program generators long ago - they didn't work. They aren't maintainable. Major updates required complete rewrites.
Now there's AI. It's a manager's wet dream...until it isn't.
...but look how productive AI is. It can whip out code as fast as a gossip can spread noise. Sure, there will be glitches, but they'll be fixed when found.
What about the $$$$$ liability of glitches that are not found?
Gerhard D.
in reply to Kevin Beaumont • • •John Breen
in reply to Kevin Beaumont • • •John Breen
in reply to John Breen • • •vlkr
in reply to Kevin Beaumont • • •encrypted.vvhispers 💫
in reply to Kevin Beaumont • • •C64Whiz
in reply to Kevin Beaumont • • •Kiernian
in reply to C64Whiz • • •@c64whiz
This was honestly my first thought.
The vast majority of the tv-news-watching public barely understands computers as it is through no real fault of their own as they have been spoonfed "magic and mystery" since the dialup days.
The distinction of "open source = MORE dangerous than big company software" would be very easy for a front of united major media outlets owned by a handful rich folks to spread and most people will not be equipped to tell facts from misinformation.
How well have those open source legal protections been working against the "smart TV" industry? I'd bet every TV holding shelf I hit at Wal-Mart will be stocked with misappropriated GPL code and no source distribution.
This is the same tactic major corps use to obtain IP for themselves.
Lock up the originator in tedious, costly busywork (typically legal, claiming infringement to start a costly time-consuming trial, for most corps) and then when the originator can't handle it
... Show more...@c64whiz
This was honestly my first thought.
The vast majority of the tv-news-watching public barely understands computers as it is through no real fault of their own as they have been spoonfed "magic and mystery" since the dialup days.
The distinction of "open source = MORE dangerous than big company software" would be very easy for a front of united major media outlets owned by a handful rich folks to spread and most people will not be equipped to tell facts from misinformation.
How well have those open source legal protections been working against the "smart TV" industry? I'd bet every TV holding shelf I hit at Wal-Mart will be stocked with misappropriated GPL code and no source distribution.
This is the same tactic major corps use to obtain IP for themselves.
Lock up the originator in tedious, costly busywork (typically legal, claiming infringement to start a costly time-consuming trial, for most corps) and then when the originator can't handle it and collapse under the weight of it all, the corps take the product as their own.
Tying up repos with vulnerabilities that might not get noticed just might work out well for the major software outfits in the long run.
It's reprehensible and a little more haphazard, but it sure looks awfully familiar.
Mal 甄/kalessin/Peri
in reply to Kevin Beaumont • • •synlogic4242
in reply to Kevin Beaumont • • •Eddy Jansson
in reply to Kevin Beaumont • • •Fortunately, I can choose to not engage.
Todd Knarr
in reply to Kevin Beaumont • • •Atomic Orbitals
in reply to Kevin Beaumont • • •Steve Hersey
in reply to Kevin Beaumont • • •Who is giving clankers commit privileges to their repositories? Seems like an obvious failure of project management.
crypticrainfall
in reply to Kevin Beaumont • • •gmoore
in reply to Kevin Beaumont • • •Unus Nemo
in reply to Kevin Beaumont • •@Kevin Beaumont
Claude contributes 0% of any of the repos I use. Though, I will keep an eye on that. Why are pull requests being accepted without analysis? These repos themselves are suspect that they we do so.
ndevenish
in reply to Kevin Beaumont • • •@davidgerard I asked it to put an OIDC flow into a confidential app. It worked! I mean, it also sent all of the secrets and access keys via the client… but someone not paying attention would probably just take it.
We’re going to see the dumbest security issues of our lives in the next couple of years, aren’t we.
Unus Nemo
in reply to ndevenish • •@ndevenish @David Gerard @Kevin Beaumont
Dumb security issues do not happen when poor code is injected into projects. Dumb security issues happen when pull requests are accepted without vetting. Keep in mind that humans have deliberately and accidentally introduced security issues into code bases far before AI.
You might rationalize that anyone can fork a repo and then push to it all they want, and it will have its own git repo online. GitHub and GitLab tell you were the repo is forked from. When I fork a repo for personal use I only fork the original project (if it has not died and been passed on to another maintainers repo). It is not a good idea to use anyone else's repo that is not in sync with the official repo. That is akin to using softwa
... Show more...@ndevenish @David Gerard @Kevin Beaumont
Dumb security issues do not happen when poor code is injected into projects. Dumb security issues happen when pull requests are accepted without vetting. Keep in mind that humans have deliberately and accidentally introduced security issues into code bases far before AI.
You might rationalize that anyone can fork a repo and then push to it all they want, and it will have its own git repo online. GitHub and GitLab tell you were the repo is forked from. When I fork a repo for personal use I only fork the original project (if it has not died and been passed on to another maintainers repo). It is not a good idea to use anyone else's repo that is not in sync with the official repo. That is akin to using software from just any download site on MS/Windows, it is asking for issues.
This is just my take on the situation. There are always going to be security issues. Our best line of defense is being aware of what we are doing. Using good OPSEC and DEVOP. There are many features that are available on modern repo servers. Such as commits being signed to verify the person that committed them (Supported on both GitHub and GitLab). If you see that a project is not using such features, consider doubling down on your due diligence.
morry040 reshared this.
Cassandrich
in reply to Kevin Beaumont • • •Unus Nemo
in reply to Cassandrich • •@Cassandrich
I agree with your concept as being a noble idea. I just do not see it as a realistic solution. These are my issues with your idea, and you may not agree with me that if fine. Your idea is that we make tools to scrape repos on git servers (and perhaps SVN as it is still used) and validate that it is accepting pull requests from AI. If I have understood you. My take on that is that if you are working on a project then you should be forking the main repository not some other person's random fork. Main repositories tend to be a lot more responsible in who they accept pull requests from. In any of these Claude infested repos was even a single one the projects actual main repository? I would guess no. If developers are practicing good OPSEC then this is a none issue. So we are adding strain on servers that is simply not required.
As developers we have a responsibility to our own integrity and are users to be sure that what we do release is as secure as we can make
... Show more...@Cassandrich
I agree with your concept as being a noble idea. I just do not see it as a realistic solution. These are my issues with your idea, and you may not agree with me that if fine. Your idea is that we make tools to scrape repos on git servers (and perhaps SVN as it is still used) and validate that it is accepting pull requests from AI. If I have understood you. My take on that is that if you are working on a project then you should be forking the main repository not some other person's random fork. Main repositories tend to be a lot more responsible in who they accept pull requests from. In any of these Claude infested repos was even a single one the projects actual main repository? I would guess no. If developers are practicing good OPSEC then this is a none issue. So we are adding strain on servers that is simply not required.
As developers we have a responsibility to our own integrity and are users to be sure that what we do release is as secure as we can make it. There is no such thing as completely secure software. It does not exist in reality (well maybe 'Hello world' 😉).
It is easy to get upset at such events. Though in the big picture is not a real issue, it is one of those issues that will be self-healing. I do not know a single developer that would not check who commits, are they using security measures like commit signing, is the project secure as is. Before forking, if they wanted to use it as a base and it did not meet those criteria they would hard fork and not participate in the original repo. Keep in mind that there are projects out there entirely written by AI, I do not endorse them, but they do exist.
It is okay to not agree with me, I am okay with that. I do not feel as if we should be censoring source code for developers. I feel like we should be teaching them about good OPSEC & DEVOPs instead. Just my opinion.
Have a great day!
Cassandrich
in reply to Unus Nemo • • •like this
LisPi and Éris Serène like this.
Unus Nemo
in reply to Cassandrich • •@Cassandrich
I do not think you read my comment, that is fine, I am not going to say that I agree to disagree with you because I never even broached the topic you responded with at all. Take care and have a great day, I can see this conversation is going nowhere. That is fine, we both have better things to do. 😀
Pseudo Nym
in reply to Kevin Beaumont • • •Reminds me of this paper from a year ago.
arxiv.org/abs/2502.17424
LLM trained (fine tuned) on code with security vulns, but not told it was vulnerable code, not only reproduced vulnerable code (expected) but also showed spontaneous ethical misalignment "judgment" in other domains.
It's a really interesting read.
If the model is producing OWASP top 10 errors like directory traversal, would seem likely it was trained on vulnerable code.
Hmmm.
see shy jo
in reply to Kevin Beaumont • • •Longplay Games
in reply to see shy jo • • •@joeyh Oh no
*well*
Guess I'm staying on the version I have.
Jeff
in reply to see shy jo • • •ninee!
in reply to see shy jo • • •scy
in reply to Kevin Beaumont • • •But think about the AI-powered "security researchers". They can now use their AI models to find these vulnerabilities and create 8.2 severity issues to fix it again.
It's like that picture with the circular economy between Nvidia and OpenAI and Microsoft, but with 0days!
Wouter De Borger
in reply to Kevin Beaumont • • •random thoughts
in reply to Kevin Beaumont • • •JTI
in reply to Kevin Beaumont • • •Eh, infinite job security I guess? (Nobody talks about pleasant jobs, just secure ones here 😆)
Peteypetepete
in reply to Kevin Beaumont • • •Laura
in reply to Kevin Beaumont • • •elle
in reply to Kevin Beaumont • • •josh g.
in reply to Kevin Beaumont • • •Luna chan
in reply to Kevin Beaumont • • •Bert Driehuis
in reply to Kevin Beaumont • • •to be fair, before Claude the same thing would happen when folks used recipes off Stack Exchange without understanding them.
The atrophying of critical thinking as a result of AI usage is the final nail in the coffin.
Kierkethumbs up convincingly
in reply to Kevin Beaumont • • •Adrian Sanabria
in reply to Kevin Beaumont • • •njsg
in reply to Kevin Beaumont • • •♾️🇺🇦 Vote Midterms
in reply to Kevin Beaumont • • •Frost, wolf of winter 🐺🎄
in reply to Kevin Beaumont • • •konstruct-960T-BF64.gguf
in reply to Kevin Beaumont • • •Cav
in reply to Kevin Beaumont • • •solo
in reply to Kevin Beaumont • • •I wish more of these fucking vibe coding agent bullshits used the co-authored-by tag, so that
Glitzersachen
in reply to Kevin Beaumont • • •Oh, man. Oh, oh, oh.
But as soon as somebody blocks every PR from AI or created with AI support the wailing starts.
Jos 🍉
in reply to Kevin Beaumont • • •Nullstring 🏴☠️
in reply to Kevin Beaumont • • •