Software is catching up to the "add more hardware" mantra that currently powers the exorbitant compute/power demands of "AI". You can now run 284B DeepSeek-V4-Flash (< 59 GiB RAM) & 30B MoEs (up to 111 tok/s) at 256K Context on Pure CPU — Zero GPUs required. And trust me, these optimisations will continue to bring down the hardware demand, resulting in even more problems to justify all those datacenters.
0 Boosts
0 Likes
@lasombra_br The current frontier models are themselves distillations of training data collected in suspicious, sometimes illegal ways too, so I don't see a problem here.
@jwildeboer Agreed, but the law is mostly on their side and governments are defending their work as they buy the hype.
There’s no problem, but we need to stop training frontier models as we do it today.
Ah, I should have added that these optimisations are not treated as super-secret, proprietary knowledge, it is simply shared in the open (in this case under the Apache 2.0 license).
@jwildeboer Although I agree 100%, these models AFAIUI are a product of distillation of the current frontier models, as it is at the moment.
Once we learn how to train more improved models without this ungodly power demand, then we’ll be in a good place.