I decided to delete a recent link I posted titled “Software is breaking down at a systemic level”, not because the content of the piece wasn’t compelling, but because after allowing it to sit in my mind for a while, it proved not sufficiently decisive, in a way that I now think is insidiously harmful.
The basis of the essay was an almost-compelling article by Kristen Maevyn called “The Decoherence Trap”. In it the author eloquently describes what happens to complex systems (software) when maintenance is performed in a way that adds “decoherence”: an emergent feature in which changes are applied to make features work, but without considering its effect on overall system complexity. It is possible, the author posits, that #AI coding would drive software to a point of decoherence beyond human comprehension, in which case we would be unable to maintain it (even though some AI may be able to).
…
But the unsaid assumption underlying both essays is that the emergent phenomenon of decoherence is not inherent in #AI itself, but in our eager overuse of the technology—in other words, we are “holding it wrong.” And the implication is that if we were to hold it right, then AI use would *not* lead to decoherence at all.
And that, I think is a fatal flaw. I believe AI coding causes decoherence by its own nature, because it invariably rewards the behavior that leads to decoherence, which are: 1. More rapid change, 2. More emphasis on shipping features over reducing complexity, 3. Increasing use of AI to comprehend code, leading to 4. Reduced expertise in human maintainers through attrition, boredom, or overwhelm.
A discussion of rapid #software decoherence due to AI *must* conclude that AI itself is an accelerant of this phenomenon, and not misuse of the technology. Although individuals may be able to pull it for a while, there is no use of this tech that doesn’t lead to decoherence *at the system level*.
…