Every OSS project maintainer I hear from is like "we can't ban AI because it's an unenforceable policy. There's literally no way of knowing whether something is written by AI" and every OSS project contributor I hear from is like "yeah, contributing is really miserable because I'm so busy reviewing this PRs and every single one is so obviously AI"
4 Boosts
3 Likes
@mcc What happened is that AI boosters brainstormed arguments that were strong enough to manipulate some people, and now everyone repeats it.
@mcc
you're listening to the wrong OSS project maintainers - there's also many who are rejecting AI and acting like everyone has already given up makes it harder for them to defend their position
@Doomed_Daniel i want an operating system and a web browser and a text editor. i want to use the "rust" and "python" programming languages. of these the only thing i can have right now is classic vim (?), but i haven't had time to install it on my computer as i've been too busy trying to replace my mail provider and maps provider and my translations app which all got replaced with gemini. then i'm gonna be real busy trying to figure out how many years until servo and redox are daily driver ready
This federated comment is not yet available here.
@aeva@mastodon.gamedev.place @moody@hj.9fs.net @mcc@mastodon.social I think a lot of people don't want the responsibility of judging "does this code smell like it was written by an LLM? Am I being paranoid and is rejecting this going to upset someone because I'm wrong? " plus questions like "we accidentally accepted a bunch of LLM authored commits 6 months ago, what do we do?"
Especially if in the latter case you might end up with an angry mob equipped with pitchforks whatever you do "It's slop now!" vs "You removed that feature I really loved"
@erincandescent @moody @aeva @mcc in practical terms, if we ignore the purity test part (which gets icky), a lot of maintainer burden can be eased by a blanket policy of "giant drive-by PR" -> instant reject
@mcc lets just have no rules for anything since people can always lie and commit fraud! this is a very sensible standpoint
@eniko "no no it's important we continue to have other rules we're just going to draw a dotted line around this one rule i want to break and say that's impossible"
@mcc @eniko our current company policy is that an engineer is the author and responsible person for the code that they submit for review, no matter how it was created. So if an engineer submits a big steaming pile of crap, we refuse to review it, and ask them why they think it's acceptable to treat their peers in such a disrespectful way, and why they think this mess is something we as a team want to maintain over the next years.
Other engineers use LLMs but adapt the code and commits into readable and idiomatic changes. That's fine. Those reviews are smooth.
@mcc Ultimately an admin shouldn't care about AI, only code quality and manageable submissions both issues perfectly solvable. The fact that solving them also takes care of AI slop is just "free" bonus. Making the issue strictly about AI is the wrong way to go even if we agree that AI is definitely the main cause of the problem. If a human post code produced by an AI, you can tell the if code is bad but never that an AI wrote it with any certainty. Sloppy developers are not something new.
@jeantranscene i think ai should be banned even if it hypothetically produces human-indistinguishable code because there are moral dimensions and externalities.
i also think this rule is probably procedurally needed because slopmongers tend to post slop and say "you should accept it because it's indistinguishable from human code! i reviewed it after all"
@mcc@mastodon.social
loved reading that coming from the most ai-in-pfp person i have yet to see on mastodon who is probably writing all their posts with ai. so cool! love that. so normal
@mcc That's a political statement and you are perfectly entitled to believe it but this have nothing to do with code repo management efficiency which is the point I was trying to make.
@mcc Ok, as a maintainer I can attest that I have started ti deny. I have accepted some ( claude ) in the past. I'm still cleaning up. Good experience, but not one I wish to repeat. I maintain 50+ programs & libs and at the moment, I just doing crack to catch up.
@mcc I think banning AI is the wrong way. We should just ban idiots. Unfortunately chatGPT makes idiots think that they know something.
@mcc
Maybe an unusual POV, but I'm fine with these PRs. I can review them only as much as I wish, and I can ignore or reject them at a whim. In the current state of affairs any formal policy looks like an invitation to fight over it.
I'm trying to be nice though.
@iorsh I would simply ban anyone who wants to have a conversation about this. They've just raised their hand as someone who is likely to try to sneak around the policy
@mcc
My feeling is that mass bans very much invite for more whining and brigading. A dozen of overblown PRs quietly rotting in the queue - much less so.
@iorsh As a contributor to open source projects, when I see a lot of ignored open PRs I worry that the maintainer is checked out. I think this is overthinking it. I've never had a high-engagement github project, but in other online situations I find people get engaged when you *argue back*. But block and delete and brickwall them and they get bored at the lack of stimulus and troll someone else.
This federated comment is not yet available here.
@mcc cowards! I make at least 4 unenforceable policies a day!
/me is a libre/free open source maintainer who is so old, he barely survived the last so-called "A'I' Winter" and was tasked with writing anti-A"I" code on others' systems in the 1980s. That it has reared its ugly head again, just speaks volumes about the collective memory chasm in this field and how the rich and powerful and greedy punch down relentlessly even when some survivors, do know better already; and avoid it like the plague.
sigh the distressing part about people having a choice is: invariably, many choose poorly.
@mcc fundamentally it feels like the problem is that most programmers view themselves as inherently logical but this response is emotional. most maintainers just want to avoid any personal friction, and refusing ai contributions is pure friction— it’s just too easy for advocates to spam and brigade any sufficiently high profile projects that deign to take a stand. but it’s embarrassing to publicly, or even consciously, admit you’re afraid of a personal cost
@mcc
Ohhh I'm so sick of seeing this one. Like, yeah, you're right. It could slip in. But you know, maybe *at the very least* reducing 98% of it and shaming the remaining 2% is infact a good idea? At that point it makes dealing with the rest of the problem a whole lot easier.
@mcc The same idiots that spend years arguing with people online about software licenses also somehow simultaneously believe that banning slop is unenforceable. It's truly a mystery.
@mcc this is changing quickly, but you can still "sense" whether something was created by an LLM or not: they completely lack a sense of "elegance". In general terms, their output feels brutish. That's the reason why enforcing such a policy is hard: "the contribution must be elegant" does not translate well.
There are some papers on using deep learning models to output a probability that a piece of code comes from an LLM. I don't know if people who do not want those kinds of contributions are willing to use models to detect them.
@mem a good thing about your criteria is you don't have to worry about false positives. you can just reject stuff like "sorry, this patch is big and the code is not good and you can't clearly answer questions about it"
@mcc paraphrasing:
"we can't have a policy against trolling, derogatory comments, public or private harassment, and other inappropriate conduct[0], because it's an unenforceable policy. there's literally no way of knowing whether soming is inappropriate"
also
"we can't require attribution in our license, because it's unenforceable. there's literally no way to know if someone used our code without attribution, and no way we can stop them from doing it"
[0] contributor covenant code of conduct a.k.a. the "we have a coc" checkbox
@mcc@mastodon.social ah yes, the two genders
@lumi possibly literally accurate
@mcc part of me feels like this is just the extremely perfect-as-the-enemy-of-the-good Software Engineer way of looking at problems. like the VLC guy saying "seeking by one frame backwards is difficult or even intractable on some worst-case video formats, so until we have a solution that always works all of the time in all situations, we will never support the feature ever"
@jk @mcc or like, "actually this is NP hard so there's no reason to even try". Look, that's not what that means, you almost never need an ideal solution!
"Oh no that's basically travelling salesman"
Hey dumbass it's like traveling salesman but the salesman is clark kent and it doesn't matter if they retrace their steps, greedy solutions are fine, just try it.
@jk i would be ok with a back one frame that took a second or so to work. or that printed an error if it detected it was going to take 30 seconds
@mcc I maintain HardenedBSD. I also maintain that HardenedBSD will not accept AI/LLM generated patches.
The only problem we have is that we maintain our patches on top of our upstream (FreeBSD). It's possible that an AI/LLM generated patch would get accepted there and be brought into HardenedBSD that way.
I'm under the (currently unfounded) suspicion that OpenZFS might have already accepted at least one AI/LLM generated patch. Those patches could be brought into FreeBSD, too, which then obviously dominoes into HardenedBSD.
It really is a tough problem. I myself don't want code that is potentially tainted with regards to licensing and IP rights.
@lattera@bsd.network @mcc@mastodon.social scrolling through the history. most of the ones i am seeing only touch code related to tests. i know some people consider that acceptable.
@lanodan @lattera @mcc @linear Yeah. If they want to go back and strip out the slop taint in tests, they can just rm all the affected tests. Zero impact on actual functionality. And until then, the potentially-dangerous code doesn't end up running on users' machines. Just git forge CI infrastructure. 🙃
@lattera Yeah, this is why i'm looking at Redox. There is a long list of reasons why Redox is not an adequate replacement for FreeBSD, but for my purposes maybe I can get away with it.
@mcc *offers bat* There seems like a simple resolution to the second one!