RE: https://mastodon.social/users/jcoglan/statuses/117546100280123064
This is an excellent thread and I recommend reading the whole thing. I'd like to say that it's getting at why competent software developers are implementing gradient ascent while LLMs+test suites are implementing generate-and-test or hillclimbing. People who know and understand a codebase are able to make changes to it that have a high probability of being improvements according to however you measure that. Stochastic code generation filtered by a static test suite is expected to be orders of magnitude slower at achieving the same improvement. This is a fundamental GOFAI result (the "knowledge-search tradeoff" or the "representation problem") but it applies to a wide swath of optimization problems.
Let's also not forget that optimization generally can only occur in small worlds, while humans---software developers and users alike---exist in large worlds. Databases, physics engines and other simulators, and research tools all have an open-ended space of possible applications ("extension") that cannot be anticipated, let alone tested for, in advance. Understanding the intended inner workings ("intension") of such systems is crucial to building them in such a way that they can support and continue to grow with their evolving applications.
#AI #GenAI #GenerativeAI #LLM #coding #software #dev #tech
0 Boosts
0 Likes
@benchsmoke@mastodon.social There are applications where no such test suite can exist, so the gap as you call it can never be filled. This question is therefore a bit like asking whether adding more epicycles closes the gaps of Ptolemaic astronomy. The answer in that case is maybe, depending on the application, but Copernicus showed us a better way to think about astronomy that has led to superior understanding, theory, and practice.
@abucci The hillclimbing comparison matches how the loop feels from the outside. Do you think a much stronger test suite closes that gap, or does it mostly depend on the person knowing the codebase?