Basically the point at which I went from "GenAI is a technology with negative effects, it should be resisted" to "delete the models, delete the software that runs the models, destroy all digital or physical artifacts ever made using the models" was when someone posted on here showing there's a command line tool in new MacOS that drops you into a chat prompt with the "Apple Intelligence" model, and in the screenshot they asked it who invented Hypercard and it said "Steve Jobs"
@mcc Sounds like someone trying to describe George Lucas with the John Company meme.
@scottmichaud @mcc I think you confuse him with the real founder of Mastodon, John Mastodon.
@scottmichaud The name sounds somewhat similar, in a way that might ??? be possible to get from the fragment-pasting thing transformers do, to the CEO of Apple who started in September 2026
@mcc is it possible that the computer was reading about Atkinson's acid trip when he designed it and went on its own trip? 🤔
@mcc that tool only uses the local only model with very few parameters and a tiny context window. I wouldn’t consider that representative of AI at all.
@collin If it's not appropriate to use the tool for the purpose, then why expose a frontend to use the tool for the purpose?
I'm not really interested in the best-case or even median performance of an LLM. I'm interested in the harm done. A lot of the harm comes from shoving LLMs into things indifferently. Some tools may be more likely to supply answers to some queries. But they're all lying just as confidently, so an LLM user doesn't know if they're using a likely-right or likely-wrong model.
@mcc I just think saying we should throw out all AI tools and everything made with them because a command-line tool you have to use sudo to run the first time is kind of crappy at some things is a bit extreme. It works really well for things like the proofreading function to fix grammar mistakes. My impression is that it’s exposed in case you have a shell script that’s a good fit or its limited capabilities.
@mcc Apple is often willing to have developer tools be a little rougher in places because the people using them are savvy enough to grasp the limitations and how to get something out of it is my impression.
@mcc @davepolaschek if you ask the more capable models meant for this kind of knowledge based question, it answers correctly.
And then the user asked the question a slightly different way, as if to prod it into giving a different answer, and it replied "Steve Jobs and John Lucas" (I still as of now am sincerely not sure who John Lucas is, all I know is that he is not Bill Atkinson)