The news that there's an oversupply of Nvidia GPUs convinced me that the AI bubble is bursting and it'll burst *hard*. So here's a thread where I go over the financial, technological and hardware implications of this. Strap in because this is going to be long and probably a bit rambling too. 🧵 1/
5 Boosts
0 Likes
If Nvidia GPUs are in significant oversupply as was reported then it means the prices will come down, as there's nobody willing to buy that stuff anymore. And this poses an immediate and very tangible problem for those who bought them: they have to write down the loss of value for the stuff they already own. That's hundreds of billions of dollars of value evaporating, because that value only existed if these things were really scarce, which they aren't. Oops. 3/
@gabrielesvelto it’s like DeBeers manufacturing diamond scarcity by hoarding, when there is an actual abundance.
Which brings to another very thorny topic: mark-to-market accounting. Let's say you buy something worth 1 billion $, say DRAM, that's 1 billion $ of assets on your balance. But let's say that the next year the price of that stuff goes up by 100%, what are you going to do? The prudent choice is to leave that asset at the value you paid it for, in case it comes down later. And then there's the choice of changing the value of that asset to account for the increase in price, that's mark-to-market accounting. Enron became famous for this, I guess you can see why. 4/
I don't know if hyperscalers did it, but given everybody involved in the AI bubble has been reckless and dishonest my guess is that it's highly probable they did. It's also considered a generally accepted accounting practice (GAAP) these days. Anyway, if they did it it means that when the prices of this mountain of hardware come down the companies owning it will have to do gigantic write downs on their balance sheets, booking huge losses, much larger than if they hadn't. 5/
But wait, it's worse than this because none of these companies had the money needed to buy this hardware. They either used venture capitals or made loans. Hundreds of billions of loans. And what does a company use as the collateral for a loan? Its own assets, right? Like GPUs. And memory. And the datacenters they're in. Which all of a sudden are losing value. And what do you call a loan whose collateral is insufficient to cover it? An underwater loan. And what does it cause? A margin call: the debtor must either deposit cash or provide a different collateral to compensate. I guess you already know where this is going. 6/
Every single company in the tech space has negative cash flow. That is they're spending more money than they're making. A lot of it. So how they'll answer the margin call? If this is starting to sound a lot like the subprime mortgage crisis it's because it very much is, but with datacenters instead of houses. 7/
This is where we have to thank Ed Zitron for its excellent reporting on the finances of these companies and his remarkable work in connecting the dots whenever some CEO talked bullshit to investors while the rest of the press was looking the other way. This is also why the Gamers Nexus investigations about GPU smuggling where so important: the demand could not slow down, by all means necessary, so a little illegal smuggling on the side is OK, say a few tens of billions of it. 8/
And this is when we start talking about the tech itself and why it is unsustainable. Every single company in this space is losing money, every AI startup, every AI division in an existing company, everybody. They're losing money because the costs of running an LLM - inference - are large, and it's unclear if anybody has been profitable there. But even if they are nobody has been able to cover the escalating costs of training LLMs, which is of course a prerequisite to running them. 9/
Training LLMs requires a gigantic amount of hardware, brute force calculations, storage, memory and power. Enormous amounts of data whose scraping is not being paid by the companies involved, it's a cost they've spread over the entire internet. As models get larger training gets more expensive. But even as companies sink more money on training LLMs have already reached a plateau in term of functionality and in some cases are degrading in a process called model collapse. 10/
Let's start with the oversupply and what that means for everybody involved. Whether it's traditional players like Google or new entries like CoreWeave every single company in the tech sector has been buying an increasing amount of hardware at increasingly steep prices. GPUs, memory, servers, power supplies, and the prices went up and up and up. Now when a company does that, it looks neutral on their balance sheet, because the money they spend is compensated by now having an asset of the same value. Then over the following years they write down that value in what is called depreciation, this is visible as an expense on their balance. 2/