I'm pretty sure that if you're using an LLM to implement something you can't call it cleanroom. The LLM has seen more code than you ever will. Probably a bunch of stolen code it had no business reading. Like I thought the idea with cleanroom impls is that you have people who produce a spec, and people implementing the spec, but they cannot be the same and the person implementing it cannot have read the source of what's being implemented. But like... it's pretty likely the LLM has. With people you can be reasonably sure that the implementing party hasn't read the code, but with an LLM you really have not a single clue whether it has or not.
@miki hmm, not sure. Couldn't someone sufficiently motivated trigger an audit of some kind? So it's a risk either way.
@talon What I mean is that it had no way to have seen that code during training, so if you have one LLM write the spec, a copyright lawyer read the spec, and an appropriately-airgapped LLM reimplement the spec, you should be fine I think. And yes, I am aware that most people are unserious about this whole enterprise and skip step 2, even though it is, and has always been, integral to the processs.
@talon The LLM wrote that but yeah, agreed.
@talon Just put "Do a clean room implementation" into the prompt or skill file or whatever these things are called and it's fine.
@talon I agree that it's very likely the LLM has seen more code than I ever will, but I don't agree that therefore it has probably seen the specific implementation I'm basing something off of. It's possible, absolutely. But it's also possible for a programmer with 30 years of experience to create the same or similar implementation to one that already exists just by virtue of 30 years of exposure to that coding environment. You're taking that programmer at his word that he hasn't seen the code previously in the same way you're taking the LLM. Why and how is one more trustworthy?
@talon That's only for OSS reimplementations, if you're re-implementing proprietary software with that technique, that's different, no? Unless it's Apple's software, at this point I'm convinced that Claude must genuinely have reversed a good bit of that in RL.