Going to try to make some small UX patches to qemu. They actually have a policy banning submitting code written by "AI" but it sure does feel a bit 😬 that the entire time I have this thing open in Sublime the upper left corner of my screen looks like this
0 Boosts
0 Likes
@mcc without looking at the source folder, I'd hazard a guess that they were written before AGENTS.md became a de facto standard that they actually follow across the board.
@trechnex That's a good guess, but my sense is that COPILOT.md and CLAUDE.md appeared first, THEN AGENTS.md, THEN after AGENTS.md was an established standard I've started to see "skills files" crop up VERY recently only. Presumably if I looked up what a "skills file" is I'd understand quickly but I am not sure I really want to know
@mcc this is something i really want to build for ngscopeclient
@azonenberg just write it! sit down with something like Sublime Text or the os x "finder" that lets you click a > to expand a folder, look at the first two levels of the folder hierarchy and write down what each thing does. then look at what you wrote and ask "how much of this is obvious? how much would be useful to leave around for other contributors?"
or do you mean you want ngscopeclient to learn to generate them somehow lol
@mcc it's not folder hierarchy, that's pretty simple (although adding more levels might help).
What needs writing is basically the internal architecture, like how the various components in the data model relate to each other and what they do etc.
Not like api documentation but a step or two above that.
And some guides on how to write a new protocol decode, instrument driver, etc
@mcc oh, I do this with my own development documentation so I can find my way around. Markdown files in obsidian, with a navigation section at the top of a unifying `start-here.md` file with "You have -> Go here...". Definitely saves enormous time.
@GavinChait shocking how few programmers realize that writing down something that helps a new-to-the-project contributor is going to help *you yourself* after you put your own project away for two years and come back to it
…that's the exact kind of information in these skills/claude.md/agents.md files?
Realizing with sinking horror it's actually in my interest to read these files. But inexplicably there are five different files, implying somehow Claude, Copilot, and Gemini all prefer SLIGHTLY DIFFERENT massaged text, and I need to read all four files to understand the actual shape of the repo, and in the process I'll come to indirectly understand the nuances between those four products. Ugh
@mcc in this case it looks to me like the separate claude and gemini files are all just pointing agents at the same underlying AGENTS.md file, which mostly says "hey AI contributions aren’t allowed upstream; don’t get your user banned", along with a skill file describing some high-level "here’s where the docs are" stuff
so the actual model-facing text is the same for all of them, and the per-vendor bits are just harness configuration
@rcombs I reckon that makes sense.
As someone who hates when you open a open source repo and there are 19 top level files and you feel lost, I do kinda feel frustrated that the github page now naturally shows you this long list of dotfiles before the README (I also have a bugbear that forges should give you more control over the appearance of the landing page)
@mcc Our agents.md is a better style guide than our actual style guide and it drives me *nuts*.
@mcc AI users have very strong superstitions about how to properly prompt their preferrred... pmodel, but i am not convinced there is any truth to it
Wait, hold on, if the project has five different files containing verbose human text that are are largely duplicative, does that implies the skills/claude.md/agents files were themselves written with genai
@mcc yes, they often basically serve as a "cache" so agents don't have to re-ingest all the same stuff to figure out the basic shape and structure of the codebase
@mcc It is very common to write these files by hand initially but then have the tool update them after any sufficiently complicated interaction.
It offends me but descriptively it works pretty okay-ish.
@mcc To my understanding, the workflow for people who have some programming knowledge often starts by having a conversation with the chatbot specifying the structure and requirements of the user. So yeah, those files are almost certainly a genAI compilation of those conversations mixed in with things that were referenced as external sources of those requirements, and any planning done in that phase.
@mcc my understanding (based on very limited testing a while ago) is that if someone uses an LLM agent, its default behavior is to edit the AGENTS.MD/etc. files constantly. It is very likely the majority of those are LLM-generated
@mcc … all the way down? At least that's how it sometimes feels like atm.
@mcc yes, absolutely. Most “skill” files I’ve seen are very much slop-generated. But, alas, they do tend to be somewhat useful to look at, since they tend to have quite detailed and recent info.
@mcc Quite possibly… the machine is documenting what it has produced.
@mcc guilty as charged, and I have no idea what to do about it :-(
@glyph As penance, immediately open a text editor and create the following two files in your Twisted working copy:
- contents.txt: Toplevel directories and important subdirs and a human readable description of each. For example https://github.com/mcclure/emily/blob/93b42a12fd8fed4c54089d55eaa889d8b7896d86/contents.txt#L9 or https://github.com/mcclure/lovr-ent/blob/d5a1e552e64d661c0656a628552c765be4ef0506/lua/contents.txt#L74
- run.txt: What commands do you actually have to type into Bash to create a local deployment, to create a package for distribution, or to run the tests? In python the first may be trivial, but I doubt the third obvious
@mcc
If you're doing run.txt, why not https://mise.jdx.dev/tasks/ with comments, to make them actually executable?
@glyph
@mcc The README includes some of these things, but others of them are torturously complex to write down, particularly because "a deployment" is such a flexible concept in this context that it dissolves into meaninglessness
@glyph Here's an example of what I mean by run.txt
https://github.com/mcclure/rs-hello/blob/3a0cb370e9e46a5d3689a5eca6d1285b71f1716a/run.txt
Often one spends so long on a repo one finds it "obvious" that (cd site && python3 -m SimpleHTTPServer) is how you boot a local test, but it's very much not obvious to someone who hasn't worked on the project before.
@glyph You don't need to actually commit these files, but if you write them and look at what you write you may be surprised how nonobvious a few things were, and maybe it would make sense to incorporate what you find yourself having written as sections into CONTRIBUTING.md or something. Looking at the Twisted repo having never set eyes on it before, it's definitely nonobvious to me what bin/admin is, or src/twisted/conch, or why that _threads directory has an underscore…
@mcc I will definitely do this exercise (have you considered making a website for this exercise? it is a good idea! Reminiscent of a non-joke version of Filefile) but two of these examples are illustrative:
1. The README has a section called "What is this" that has a bullet point, "``twisted.conch``: SSHv2 and Telnet clients and servers and terminal emulators". Should we have a contents.txt that has several paragraphs explaining how Python modules map to directories?
…
@mcc Oh yes, so much this.
@mcc Surely those contain instructions for the chatbots to fuck off, right?
@mlen Well, now that you mention it, they do begin with an exhortation that the chatbot must only participate in the repo in accordance with the AI policy (which is quite strict that the AI can only be used for querying information about the repo or for non-committed experiments). But I severely doubt this is the kind of instruction an AI bot can meaningfully follow.
Thinking about it…
Few open source projects have contributor documentation, and the missing thing that always frustrates me most is the lack of "structural documentation" (I call it this, idk what else to call it). I always spend so much time trying to figure out WHERE TO FIND THINGS. I want to know where in your directory tree the argument parsing is, the menu building is, the rendering code. You don't tell me and most project src/ trees grew organically and not according to any true logic.