12 Boosts
2 Likes
This federated comment is not yet available here.
I will continue uploading to #Debian NEW, but I will most likely not read any reaction to an upload, that has been written solely by an #LLM.
I love the Debian project for #human interactions. Let's keep Debian human or at least nice among #primates, #naked or #hairy.
While I'm personally very critical about use of LLMs, I do accept, if people use it to e.g. polish their english or similar things. But don't throw unfiltered AI output at primates. We deserve better 🙂
@debacle @spanezz @olasd @josch
I find pretending to answer to an upload in NEW queue solely by LLM unattended tooling is something to discuss, not automatically adopt.
It is a tool, as for lintian, not a team decision. As such, it should help the team, with a suitable review. Possibly, also the maintainer, but in both cases it cannot be considered an obligation.
@gisgeek it is a tool designed by technocratic fascists that is also completely unfit for any purpose and causes massive amounts of externalized harms. it should not be used at all.
@atax1a well, specifically that service is based on Chinese open weight models and hosted on Scaleway in EU, an OVH subsidiary. So, ok maybe you need to adjust a bit your trolling experience. 😎
@atax1a @gisgeek which models? it is alleged, for example, that Qwen has really bad alignment problems, which has been partially validated through misalignment research.
also, bluntly, anything that is connected to the CCP (which anything AI-related will certainly be in its orbit) is still fascist, albeit with Chinese characteristics
@atax1a @gisgeek @neko sure but my point is that using and thus becoming dependent on Chinese models is probably at least as harmful to Debian as becoming dependent on proprietary models as there have been demonstrated scenarios where Qwen and GLM are far more likely to generate code with security vulnerabilities if it concludes the code is for organizations the CCP does not like.
and if you make some minor tweaks to the weights it stops doing that.
so in essence I am saying that the misalignment issue previously mentioned is a security risk to Debian.
@gsuberland @ariadne @atax1a @gisgeek @neko I'm also curious about this. I've warned people against this in theory, but I hadn't heard of any real world validation. Would love to have references to this that I can pull out!
@ariadne @gsuberland @atax1a @gisgeek @neko @notthatdelta
"The threat is not an obvious backdoor in the code. In fact, we do not have proof at this point that code flaws are intentionally introduced."
recommendations:
1. Ban Use of Untrusted AI Models for U.S. Government and Critical Infrastructure
2. Invest To Make Trusted American AI Models the Global Default
Sure...
@gsuberland @atax1a @gisgeek @neko @notthatdelta @ipaschke that's booz allen, a US defense contractor. they are going to say that.
@ariadne @gsuberland @atax1a @gisgeek @neko @ipaschke Yeah, I think the takeaway here is that if Chinese models could have baked-in rules to generate less-secure or exploitable code, there's no reason that US models couldn't have similar baked-in rules.
I guess I'm approaching it as more of a "here's yet another reason you shouldn't trust any model for anything", myself. They're going to be biased in some way, either intentionally or unintentionally (based on the datasets used to generate the model).
@gsuberland @atax1a @gisgeek @neko @ipaschke @notthatdelta that is indeed my goal in bringing up the research! open models do not automatically mean better!
@ariadne @gsuberland @atax1a @gisgeek @neko @notthatdelta That's very true. They're all a black box even to their creators.
And I don't think OpenAI and Anthropic are doing a great job positioning their models as the safe alternative to the world with their constant stream of press releases on how dangerous their models are 🙂. "The world" might also have some specific trust issues right now.
@josch so good that i can't even read this without having javascript enabled because "AI scrapers break the web"
oh do they really, Debian? interesting. i wonder if there's anything else relevant going on with that whole situation? :/
In the ongoing discussion [1], it looks like there's rough consensus to make the LLM NEW-queue reviews at least opt-out, and preferably opt-in. It's bad that someone thought that they could impose this without getting any objections, but it looks like there are plenty of objections (only one consequentalist claim - "but it works" - that I can see).
[1] https://lists.debian.org/debian-devel/2026/10/threads.html#00263
@boud Yes, the DFSG team might implement some changes after having received quite a bit of feedback on this topic now. But even if they change it, I worry about the long-term situation. If I start having to worry "what problem will it be today?" when I wake up, then this hobby will become more dread and less fun and then I will just put my priorities elsewhere. @mildpeach @Lamuchan
@alxndr Oh dear, that reminds me of this screenshot somebody shared of a prompt they had with some chatbot where they said "no" only for the chatbot to then go on and write "you said no but i think what you really mean is..." and then went on to ignore the "no" and do the opposite... Sigh...
@josch Well, there's probably a ton of "pickup artist" forums in the training data. 😑
@josch I am so sorry Debian is going this route and that you found yourself in such position. That sucks. Hope your time outside is great (it is raining a lot here so it is time for coding with rain sounds for me)
@soapdog I have worked with computers so much in my life that more and more I'm in the camp of "maybe it would be better to live somewhere in the mountains with some goats and lots of nature around" The more I do with computers, the more I find myself learning to appreciate how friggin' beautiful it is outside and even how nice it feels to have your fingers burn when you bike through the rain because heck it reminds you that yes, you are alive and life and this planet are so amazing. :)
