i think i could write a really good article about how the "documentation" in slop project repos is at best useless and at worst an actively harmful abomination. BUT that would require sustained close reading of the "documentation" in slop project repos. which i would like to avoid doing, so as to not expose myself to the risk of actively harmful abominations
@phred i feel like even really bad conventionally composed documentation is noticeably different structurally and epistemologically from llm-generated documentation-ish text. but yeah it's tough to put your finger on!
but here's a quick stab at it: (1) no one is asserting the truth of llm-generated documentation—it's impossible to know whether the text, even if generously interpreted as actual assertions (rather than statistical artifacts) does now or has ever truthfully described the software. there's a contract between the reader and the writer of conventionally written docs, i.e., that the docs are useful and true; i don't think ppl who git push slop understand themselves to be in that contract w/readers
(2) a good writer of docs has a consistent/contextual model of the knowledge the reader brings to the text (which changes *during the process of reading*) and uses that both as a yardstick for deciding when more context is needed, and as scaffolding for building new knowledge. llm-generated documentation-oid text feels more like reading someone's notes, which don't take the reader's context and knowledge into account, because only the note-taker was ever meant to read it
(3) without that model of the reader (which again has to be contextual, specific, ever-changing), it's impossible to draw attention to things that are counterintuitive, or surprising, or interesting, which of course is one of the main purposes of documentation in the first place (this thing isn't the same as what you're already familiar with; here's why, and how, and why it's important)
possible i guess that this is just careless prompting on the part of the sloppers, and/or that sam and dario are just a few gigabytes of text and a few hundred hours of reinforcement learning data away from making tech docs appear to incorporate some simulacrum of the reader's actual needs, but that doesn't stop llm-generated tech docs from being barf-inducing bad right now
(thread inspired by this project https://github.com/ccoors/dussek whose "documentation" is 100% bewildering and clearly not intended to actually be read)
@aparrish yes, definitely agree on it look like notes. So much of it could be spun off into a tutorial separate from the docs. Or, y'know, be written and read by a human
@hellocatfood i really wish the humans behind these projects would at least write the part at the top of the readme that explains what the thing is and why it's interesting 🙃
@aparrish I wish I could put my finger on what it was that makes this kind of documentation so uncanny. About halfway into a project's docs something about the writing makes my spidey-sense tingle: it's repetitive, and the information-per-sentence ratio goes down, perhaps? I know what it looks like to pad out a piece of writing.
But also, what if I'm wrong and the person is not yet good at writing technical docs?
I try to work in spaces where the documentation and code work together as a living document, but that's not always feasible.