Every OSS project maintainer I hear from is like "we can't ban AI because it's an unenforceable policy. There's literally no way of knowing whether something is written by AI" and every OSS project contributor I hear from is like "yeah, contributing is really miserable because I'm so busy reviewing this PRs and every single one is so obviously AI"
0 Boosts
0 Likes
@mcc The same idiots that spend years arguing with people online about software licenses also somehow simultaneously believe that banning slop is unenforceable. It's truly a mystery.
@mcc this is changing quickly, but you can still "sense" whether something was created by an LLM or not: they completely lack a sense of "elegance". In general terms, their output feels brutish. That's the reason why enforcing such a policy is hard: "the contribution must be elegant" does not translate well.
There are some papers on using deep learning models to output a probability that a piece of code comes from an LLM. I don't know if people who do not want those kinds of contributions are willing to use models to detect them.
@mem a good thing about your criteria is you don't have to worry about false positives. you can just reject stuff like "sorry, this patch is big and the code is not good and you can't clearly answer questions about it"
@mcc paraphrasing:
"we can't have a policy against trolling, derogatory comments, public or private harassment, and other inappropriate conduct[0], because it's an unenforceable policy. there's literally no way of knowing whether soming is inappropriate"
also
"we can't require attribution in our license, because it's unenforceable. there's literally no way to know if someone used our code without attribution, and no way we can stop them from doing it"
[0] contributor covenant code of conduct a.k.a. the "we have a coc" checkbox
@mcc@mastodon.social ah yes, the two genders
@lumi possibly literally accurate
@mcc part of me feels like this is just the extremely perfect-as-the-enemy-of-the-good Software Engineer way of looking at problems. like the VLC guy saying "seeking by one frame backwards is difficult or even intractable on some worst-case video formats, so until we have a solution that always works all of the time in all situations, we will never support the feature ever"
@jk @mcc or like, "actually this is NP hard so there's no reason to even try". Look, that's not what that means, you almost never need an ideal solution!
"Oh no that's basically travelling salesman"
Hey dumbass it's like traveling salesman but the salesman is clark kent and it doesn't matter if they retrace their steps, greedy solutions are fine, just try it.
@jk i would be ok with a back one frame that took a second or so to work. or that printed an error if it detected it was going to take 30 seconds
@mcc I maintain HardenedBSD. I also maintain that HardenedBSD will not accept AI/LLM generated patches.
The only problem we have is that we maintain our patches on top of our upstream (FreeBSD). It's possible that an AI/LLM generated patch would get accepted there and be brought into HardenedBSD that way.
I'm under the (currently unfounded) suspicion that OpenZFS might have already accepted at least one AI/LLM generated patch. Those patches could be brought into FreeBSD, too, which then obviously dominoes into HardenedBSD.
It really is a tough problem. I myself don't want code that is potentially tainted with regards to licensing and IP rights.
@jeremiah_ @mcc I've never heard of Contributors Covenant. HardenedBSD-specific code uses either the 2BSDL or the 3BSDL depending on the specific code.
@lattera @jeremiah_ "Contributors Covenant" is a code of conduct, and I'm not sure if it's relevant. Do you have a DCO?
@mcc @jeremiah_ I'm unsure what a DCO is, either. Focus has been on hacking on code.
@lattera @jeremiah_ A DCO is an agreement some projects make contributors agree to where they simply affirm that they are the author of the code and have permission to use it. As opposed to it being stolen from IBM on an usb thumbdrive or something.
@mcc @jeremiah_ Ah. We're still such small fry that we haven't had to worry about any of that.
@lattera@bsd.network @mcc@mastodon.social scrolling through the history. most of the ones i am seeing only touch code related to tests. i know some people consider that acceptable.
@lanodan @lattera @mcc @linear Yeah. If they want to go back and strip out the slop taint in tests, they can just rm all the affected tests. Zero impact on actual functionality. And until then, the potentially-dangerous code doesn't end up running on users' machines. Just git forge CI infrastructure. 🙃
@dalias @lattera @mcc @linear I think it can make sense to still have tests against private interfaces, for example to avoid lengthy tests or hacks like test-only code.
That said I think anything not-cleanly-foss (or other license annoyances) belongs into a separated repo and ought be entirely optional.
Which goes even beyond tests as most OS projects will end up with firmware blobs as well.
That said I think anything not-cleanly-foss (or other license annoyances) belongs into a separated repo and ought be entirely optional.
Which goes even beyond tests as most OS projects will end up with firmware blobs as well.
@lattera Yeah, this is why i'm looking at Redox. There is a long list of reasons why Redox is not an adequate replacement for FreeBSD, but for my purposes maybe I can get away with it.
@gooba42 @mcc Folks are finding success in running many Steam games under FreeBSD's linuxulator, which HardenedBSD inherits from. Though, intentionally HardenedBSD makes it rather painful to enable the linuxulator (it's possible--you just have to run a few extra commands to say "I know the risk I'm taking, please let me proceed").
@sackofbones @reflex @lattera Is there a reason you are telling us this? It's obvious that projects use libraries and a project's policies may not re.
The way I see it, if you have a project that's committed to no-AI, and the language or the libraries become tainted, languages and libraries are "interfaces" which can be swapped out at a later time for clean ones. The important thing is the project commit to an intent; that gives you a "safe" zone you can stand in and clear out slop from there.
The important thing is the project commit to an intent; that gives you a "safe" zone you can stand in and clear out slop from there.
I agree, but only to the level the project's maintainer(s) can and want to support. For example, I have absolutely zero time (or desire) to audit each and every commit from each and every third-party vendored code in FreeBSD. It's already difficult tracking FreeBSD src on a commit-by-commit basis. (Thanks to the FreeBSD folks for providing per-commit emails!)
(Sorry to go on a tangent parenthetical, but that last paragraph touches on something I've learned as the long-term maintainer of a FOSS project: we, as a community, need to be careful about placing expectations on how random other people on the internet use their time. Only my family, my employer(s), and I can set my daily schedule or priorities.)
So, coming from the aspect of the futility of the lack of time: the intent (for me and the HardenedBSD project specifically) to be an "AI safe zone" is there--but the ability is not 100%. My time is much better spent implementing exploit mitigations and doing other R&D than reading git commit logs.
If someone in the community wanted to take that effort on, by all means--and that's part of what makes open source great! I myself can't.
@mcc @sackofbones @reflex (I apologize for the long post--nuance is warranted here)
@sackofbones I suspect that, for example, the maintainer of HardenedBSD has enough information to figure that out themselves
@sackofbones @mcc @lattera Never let the hypothetically perfect be the enemy of the better.
With projects deciding *not* to flag LLM commits we aren't going to be able to avoid all the slop. We can, however, state our principles and do our best to stand by them.
Also uutils (1) is command line, so i don't have to use it; I can make a functioning system without a command line prompt, and
(2) they forked uutils, I don't think they're still downstream; I was involved in writing docs for them and when I asked if I should describe the core command line tools as "from the uutils project" they were like "no not anymore"
@mcc
Ohhh I'm so sick of seeing this one. Like, yeah, you're right. It could slip in. But you know, maybe *at the very least* reducing 98% of it and shaming the remaining 2% is infact a good idea? At that point it makes dealing with the rest of the problem a whole lot easier.