Since when do you buy a direct A15? Or as Tranium? Or Maia? Or cobalt? Or graviton? Or a TPU? The hyperscalers are selling to enterprises not consumers. The apple chips are sold to consumers but they sell the whole hardware not just the chip
in the Price for an nvidia card is not only the price for the card. Its also the limitations of producing it.
The Hyperscalers have much deeper pockets than the consumers, so they compete via the price to buy as much as possible from the limited batches nvidia/amd/... sell.
As a result, manufacturers are changing the number of different chip types, so that instead of 4 graphics cards and 1 CUDA card, there is now 1 graphic card and 3 CUDA cards.
My current understanding is that regular consumer GPUs are very much capable if they had enough VRAM, and buying like 2 isn't out of this world.
A 64GB consumer GPU not existing has no technical nor financial (buyer side) reason to it, other than market segregation. If a 24GB one costs 1000 bucks, people are ready to pay quadruple that for a 32GB GPU. And VRAM is the main bottleneck for running larger LLMs, not performance. And some resort to offloading to RAM etc...
Segregation between pro and consumer hardware are in a big part artificial I think, for fatter margins.
Otherwise, yeah Hyperscalers have deep pockets, but they can only have it by getting back money from their users/customers, and we're gonna give them that by using their products and paying for it (indirectly or directly).
I don't think it's a good thing that only select companies gatekeep AI and it feels to me like it's going that way with chip prices.
Since when do you buy a direct A15? Or as Tranium? Or Maia? Or cobalt? Or graviton? Or a TPU? The hyperscalers are selling to enterprises not consumers. The apple chips are sold to consumers but they sell the whole hardware not just the chip
in the Price for an nvidia card is not only the price for the card. Its also the limitations of producing it. The Hyperscalers have much deeper pockets than the consumers, so they compete via the price to buy as much as possible from the limited batches nvidia/amd/... sell. As a result, manufacturers are changing the number of different chip types, so that instead of 4 graphics cards and 1 CUDA card, there is now 1 graphic card and 3 CUDA cards.
My current understanding is that regular consumer GPUs are very much capable if they had enough VRAM, and buying like 2 isn't out of this world.
A 64GB consumer GPU not existing has no technical nor financial (buyer side) reason to it, other than market segregation. If a 24GB one costs 1000 bucks, people are ready to pay quadruple that for a 32GB GPU. And VRAM is the main bottleneck for running larger LLMs, not performance. And some resort to offloading to RAM etc...
Segregation between pro and consumer hardware are in a big part artificial I think, for fatter margins.
Otherwise, yeah Hyperscalers have deep pockets, but they can only have it by getting back money from their users/customers, and we're gonna give them that by using their products and paying for it (indirectly or directly).
I don't think it's a good thing that only select companies gatekeep AI and it feels to me like it's going that way with chip prices.