the transformer model used by LLMs is a pretty simple approach that works exactly for the purpose of its proprietors. it's a keyword search engine with an obfuscation layer attached. it is the perfect expression of what the tech sector gets all that damn money for. it's not to make things more efficient. and it identifies information architecture—library science—as the hero to fight the final boss of capital
0 Boosts
0 Likes
@hipsterelectron This doesn't quite seem right to my ears: once you hit the Pareto-optimal frontier, efficiency and resiliency are fundamentally at odds with each other.
For example, consider sorting lists of elements where the comparison function has some probability of messing up, say 1 in 100 or 1 in 1000 or whatever, and assume the probability that any given execution of the comparison routine being faulty is independent of all other executions.
Quicksort is very efficient, with asymptotic complexity at the theoretical maximum, but it's disastrous with unreliable comparison functions.
Bubblesort is often used as an example of an inefficient sorting algorithm, but it performs as well as can be expected (in terms of correctness, not necessarily time) in the presence of faulty comparisons. Unlike with quicksort, a bubblesort will approach a nearly-sorted list, with every element being almost certain to be within one or two elements of their correct positions.
What you describe sounds likely to lead to a resilient system, but not necessarily the utmost efficiency.
In my mind "efficiency" is approximately "reward divided by cost", though less careful use of the word can mean a number of other things.
So I'd agree that efficiency is closer to throughput than latency, but not really the same either because it's fairly common that the most efficient processes tend to have difficulties with throughput, and processes that provide great throughput to kind of suck in terms of efficency.
In my mind, conflating efficiency with throughput is similar to conflating efficiency and power.
@leon_p_smith you mention "faulty comparisons" but i'm not familiar with a model of faulty comparisons (which includes a hidden distributional assumption) and i would absolutely challenge your argument for its optimality
@hipsterelectron The exact distributional model doesn't much matter to the point I'm trying to make in this case, which is once something reaches a certain level of "competence", efficiency and resiliency are fundamentally in tension with each other.
The intention for the faulty comparisons example is that every time the comparison subroutine is executed on the computer, independent dice are rolled to see if the result is going to be inverted or not. But variations on this should often work just as well, as long as you don't get into comparison subroutines that consistently return the wrong result on certain pairs of inputs.
Also, you usually are never quite certain if you are on a Pareto-optimal frontier. More often it's a comparison between multiple things, some of which might be on the Pareto frontier, and others are more "obviously" not on the Pareto frontier (inferred from the relationship to the things that *might* be on the frontier)
So I'm not going to make any claims that bubblesort is on the relevant Pareto frontier, but it's not actually all that hard to see that it is "optimal" in the sense of producing a list that is (almost) in the right order. (After all, a comparison function that's right 99% of the time doesn't have any reasonable chance of sorting a list of 1000 elements...)
Similarly I'm not going to claim that Quicksort's on the Pareto frontier, but it's well known that it is asymptotically optimal in terms of the number of comparisons made.
The reason is that you count the number of orderings of a list, and then take the binary logarithm of that, and log(n!) = O(n*log(n))
It's not difficult to see that a sorting routine that uses the absolute minimum number of comparisons to sort the list cannot be resilient. If you think of the result of each comparison as being an input to a function that permutes the list, an optimal number of comparisons means you have zero error correcting bits left over.
Of course quicksort is only *asymptotically* optimal in this sense, not necessarily optimal, but it's not hard to actually run this experiment yourself and see that yeah, when your comparison function exhibits this kind of error model, quicksort doesn't do well. You get badly broken outputs, not nearly-sorted lists.
But you also don't need to know that anything is on a Pareto frontier to be able to use a couple of examples to understand that the Pareto frontier does indeed *exist*.
making things more efficient is antifascist because fascism is parasitic and ideologically opposed to efficiency. the fascist claim to efficiency is that 6 million jewish people were dead weight upon society. efficiency, as any graph theorist can tell you, arises from highly connected networks with lots of redundancy—this is not a quality of the internet