it's time to play everyone's favorite game, "Allison Once Again Tries To Use An LLM For Research Because The World Is Constantly Gaslighting Her Into Thinking That LLMs Are Suited To That Purpose, Only To (Once Again!) Have The LLM Confidently Spew Misinformation Until Explicitly Called Out"
0 Boosts
0 Likes
i really, really don't get it. the topic i'm researching here isn't *all* that obscure—it's like a semi-expert level topic, not that far from the kinds of questions i'd expect academics to research in their own fields—and not only is the LLM incapable of producing an accurate answer to the question, it generates one confident lie after another in what looks like an attempt to cover up its tracks. how can so many otherwise reasonable-seeming people use tools like this for research? it's baffling
@aparrish it's nuts to me too
(this is the "gemini 3.6 thinking" model btw, which i have access to through nyu, for some god forsaken reason)
i am being told that if i use a newer model then the meaning will, somehow, magically inhere in the text, and facts that have yet to be uncovered will spontaneously burst into the light, propelled by magnificent statistical weights alone
@aparrish My read is that the “trick” here is what makes newer models “more accurate” is mostly that they’re being fed more comprehensive training data, making the statistical inference more likely to result in a correct response. Asking anything outside of the “comfort zone” of any models training still is going to result in confident guessing.
Agreed that spending trillions to pretend this isn’t the case seems pretty foolish.
@aparrish which environment is this in? i don't use gemini much but this seems off as far as current model generations go. I'm happy to give it a go if you give me your starting prompt / intent. contrary to (im)popular opinion, there is a (more or less steep) learning curve.
@aparrish 3.6 is fairly old. I recommend trying 3.8-flash in Google AI Studio or something like deepseek. Or limited models like Anthro/OAI/etc
@aparrish it's so weird that technology has advanced to the point where we can make a word prediction algorithm that talks about lying and to no other point
(i'm trying to figure out whether resetting the div counter on the game boy also resets the phase of the serial clock pin, and each time i asked it to give me an actual source for its conclusions, it just dug itself deeper and deeper into made up stuff)