Slightly edited except from an email I just sent to my department list in response to someone complaining about LLMs "solving" open math problems too fast:
Way back in 2021, as the current AI hype cycle was just beginning, the paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" was published, and I'm fairly confident we've all read it at some point since then. Its publication cost Timnit Gebru her job, and ultimately led to the effective shuttering of Google's entire "AI ethics" division at the time. There couldn't have been a clearer signal that Google at least intended to cross some ethical lines in their pursuit of wealth. To reiterate what the paper said, it boils down to:
"Please stop building ever-larger language models. Building them causes harm."
Positioning myself as a member of a faction in the CS research community who agrees with this message, we were collectively making specific requests:
1. Building these things has negative environmental impacts, at a time when the world desperately needs positive environmental impacts. Please stop and invest that money into technologies that would mitigate, not accelerate climate change.
2. Specifically, the people who will suffer most from the environmental impacts of these models are the people least likely to benefit from them. Please don't ignore the impacts that building and using these models will have on marginalized communities.
3. Building ever-larger models with barely-curated datasets introduces biases into those models that will harm their users. It also means that nobody, not even those who supervise the training, will have full insight into what is and isn't in the training data, making it very difficult to evaluate any claims about model performance since memorization is a cofounder. Please budget for serious dataset curation and use smaller, better-curated datasets for training models.
4. The hype being generated by LLM tech demos is misleading the public about the capabilities of these models, and that's dangerous for their users. Please tone it down and be realistic. Explain what these models actually do, and warn people about their limitations.
5. Specifically, these models manipulate linguistic form without real understanding. Investing everything in these models takes resources away from projects that would actually make progress towards understanding understanding, so please moderate your investments and look beyond shallow benchmarks.
6. The biases in these models, combined with the ways they evoke misunderstandings in not just the general public but even trained ML researchers, have potential to do serous harm, including amplifying negative stereotypes and bias in our society, and automating discrimination. Please stop building and using bigger models, stop hyping them up, and start working on these problems, in part by using smaller, more focused models and explaining things more clearly.
The collective response from a different faction in the CS research community was basically: "we don't care." Certainly that was the message from the leaders of tech companies developing these models, who have leaned into these harms in a cynical race to pump up their valuations and make money from the hype cycle. That was also the message from everyone using these models, despite all the warnings about harms: "I don't care if building and using this model is harmful, I think it's useful to me, so I'm going to use it and encourage others to use it, no matter who gets hurt." Some in this group have discussed some of the ethical issues before dismissing them, but actions speak louder than words, especially when one not just uses an LLM, but recommends it to their colleagues and students.
So what is my response when informed that LLMs are solving too many math problems?
"Why should I care?"
Now I do care, and it is an issue (even if a big part of the issue is really about plagiarism). But why *should* I care about *this* harm, when you didn't care about any of the others?
@tiotasram
Also if individual researchers were given 1% of the budget and compute that LLM companies have been throwing at those open math problems how fast would they be solving them?