One question for anyone who wants to opine - can someone provide an argument as to why any of these tier 2 GPU cloud companies (e.g., Coreweave, Llamda Labs, Nebius, etc.) should exist in 5 years in the context of competing against the hyperscalers (e.g., AMZN, MSFT, GCP, ORCL, etc.)?
"I have spent some time studying Coreweave / Llamda Labs in the context of some companies who serve the datacenter supply chain and supply to these companies, and I have not been able to come up with a reasonable argument to the affirmative.
Coreweave all but admitted to me that the initial reason for their existence and success was due to 1) the fact that there was a worldwide GPU shortage, and 2) at least for the time being, NVDA is incentivized to spread GPU allocation in a way that props up these tier 2 GPU clouds so that the hyperscalers who are currently trying to disintermediate them with custom ASIC accelerators do not dominate the early stages of AI accelerated compute. That only works so long as GPU compute remains in a constrained supply environment, and particularly where NVDA remains dominant with 90%+ share.
In a world where GPU compute is no longer constrained (e.g., 3 years, 5 years from now, I don't know, but eventually), how can these tier 2 GPU cloud companies compete with the hyperscalers, and why would any customer use them? The resale of compute and storage is in some ways a commodity (setting aside the ecosystem effects the hyperscalers have via their marketplaces), and the hyperscalers have enormous scale advantages and lower costs of capital which ensures their costs will be lower. Additionally, if I am your standard F500 or any enterprise for that matter, all of my data is on some combination of AWS, Azure, GCP already. Why would I go through the enormous effort to move that data (which is needed for AI to work) to some tier 2 company to use its GPU compute capacity?
I genuinely don't understand why any of these companies have a place in the world over a medium to long-term horizon, but am looking for someone to argue the other side here."
1. I could see some undersupply if the hyperscalers get really aggressive with their in house projects and they really flop long term. They’d have to start preemptively scaling back orders of GPUs aggressively and then face unexpected friction wrt their in house ramps (the TPU family wasn’t built overnight). I imagine if that transition stretches out, and demand continues to be tough to pin down, it could be really difficult for hyperscalers to thread the needle.
2. The tier 2 players can somehow outmaneuver the hyperscalers with grid connection timelines. Sustainability commitments are probably the biggest crutch here, but maybe Tier 2 firms also find effective tactics around load flexibility and behind the fence options. If capability jumps continue and the adoption curve steepens, demand could really outpace algorithmic efficiency gains. Power bottlenecks could become more pronounced and that might open more doors to scrappiness.
Just playing devils advocate here, I share some of your skepticism.
I like your questioning and it made me think. I kind of agree with you now that you've planted that seed in my mind. But just to play devil's advocate for the sake of it, I think the play here is to be faster and cheaper than the hyperscalers very quickly. They are already cheaper than AWS for certain compute loads.
If they can suck you in, the data that you generate quickly becomes the quicksand you bury yourself in. It'll just be too sticky and difficult to migrate back out to a hyperscaler or your own on-prem infrastructure.
I'm sure there is something there, but you're right that the hyperscalers are not going to sit still while someone else is trying to take their lunch. They are going to find some way to make it easy or financially viable to migrate to their platforms.
Would be nice to have a crystal ball to tell us what happens in 5 years time.
One question for anyone who wants to opine - can someone provide an argument as to why any of these tier 2 GPU cloud companies (e.g., Coreweave, Llamda Labs, Nebius, etc.) should exist in 5 years in the context of competing against the hyperscalers (e.g., AMZN, MSFT, GCP, ORCL, etc.)?
"I have spent some time studying Coreweave / Llamda Labs in the context of some companies who serve the datacenter supply chain and supply to these companies, and I have not been able to come up with a reasonable argument to the affirmative.
Coreweave all but admitted to me that the initial reason for their existence and success was due to 1) the fact that there was a worldwide GPU shortage, and 2) at least for the time being, NVDA is incentivized to spread GPU allocation in a way that props up these tier 2 GPU clouds so that the hyperscalers who are currently trying to disintermediate them with custom ASIC accelerators do not dominate the early stages of AI accelerated compute. That only works so long as GPU compute remains in a constrained supply environment, and particularly where NVDA remains dominant with 90%+ share.
In a world where GPU compute is no longer constrained (e.g., 3 years, 5 years from now, I don't know, but eventually), how can these tier 2 GPU cloud companies compete with the hyperscalers, and why would any customer use them? The resale of compute and storage is in some ways a commodity (setting aside the ecosystem effects the hyperscalers have via their marketplaces), and the hyperscalers have enormous scale advantages and lower costs of capital which ensures their costs will be lower. Additionally, if I am your standard F500 or any enterprise for that matter, all of my data is on some combination of AWS, Azure, GCP already. Why would I go through the enormous effort to move that data (which is needed for AI to work) to some tier 2 company to use its GPU compute capacity?
I genuinely don't understand why any of these companies have a place in the world over a medium to long-term horizon, but am looking for someone to argue the other side here."
Tons of firms will want to rent and not build their own for various reasons, including Chinese companies. Def will be a huge TAM eventually.
1. I could see some undersupply if the hyperscalers get really aggressive with their in house projects and they really flop long term. They’d have to start preemptively scaling back orders of GPUs aggressively and then face unexpected friction wrt their in house ramps (the TPU family wasn’t built overnight). I imagine if that transition stretches out, and demand continues to be tough to pin down, it could be really difficult for hyperscalers to thread the needle.
2. The tier 2 players can somehow outmaneuver the hyperscalers with grid connection timelines. Sustainability commitments are probably the biggest crutch here, but maybe Tier 2 firms also find effective tactics around load flexibility and behind the fence options. If capability jumps continue and the adoption curve steepens, demand could really outpace algorithmic efficiency gains. Power bottlenecks could become more pronounced and that might open more doors to scrappiness.
Just playing devils advocate here, I share some of your skepticism.
I like your questioning and it made me think. I kind of agree with you now that you've planted that seed in my mind. But just to play devil's advocate for the sake of it, I think the play here is to be faster and cheaper than the hyperscalers very quickly. They are already cheaper than AWS for certain compute loads.
If they can suck you in, the data that you generate quickly becomes the quicksand you bury yourself in. It'll just be too sticky and difficult to migrate back out to a hyperscaler or your own on-prem infrastructure.
I'm sure there is something there, but you're right that the hyperscalers are not going to sit still while someone else is trying to take their lunch. They are going to find some way to make it easy or financially viable to migrate to their platforms.
Would be nice to have a crystal ball to tell us what happens in 5 years time.
https://www.ft.com/content/f3d9d339-42ef-4979-bf52-89ecd699dea2
Well that was quick. Microsoft has withdrawn from some of its agreements with @CoreWeave over delivery issues and missed deadlines ahead of ipo.
This will indeed be an important AI Infrastructure IPO to function as a barometer for the Datacenter capex absurdity we have witnessed.