I decided to push the limits of what I was getting an agent to do on this personal project and dear gods was that an exercise in frustration.
Even with extremely advanced tools and high level models, along with extremely stringent quality controls and a robust library of skills, it was _inordinately_ painful.
Some of the pain points:
- It's extremely well established in this codebase not to use synchronized blocks. It used them anyways on a caffeine cache
- When an adversary called it on the synchronized block it decided to write its own cache entirely and use double-checked locking.
- When I called it on that it then decided to read the entirety of the source code for Caffeine and implement something that was dependent on implementation details deep in how caffeine builds caches.
- When it was called on not using arguments it was requiring in methods it decided the right solution was to _write its own lock_ using java's built in LockSupport class. Nb. The feedback given had nothing to do with locks or locking, it was about an unnecessary argument.
Even to the degree it "worked" it was _excessively_ expensive and each of those errors required both human and machine time to fix, and those were just a few of the many, many errors I encountered while doing this.