Ben Thompson
speaker
42,103 appearances
120 recordings
2 series
first heard Sep 2024
last heard 21 Aug
Ben Thompson’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 39 in all, peaking in Mar 2026 with 5.
Appearances
Training needs this horizontal scaling, this super low latency between shifts, between systems.
podcast-host
And Amazon has never been a big player in terms of training.
podcast-host
It like
podcast-host
It's been Microsoft, it's been Oracle, it's been, you know, Elon Musk and XAI building their own data center that is dedicated to doing this.
podcast-host
And what Amazon got right is training for a long time dominated the amount of compute that was used, even post chat GPT for several years.
podcast-host
It like more like sixty percent of of chips in the world were being used for training, not for inference.
podcast-host
Mm-hmm.
podcast-host
But when and if inference came along.
podcast-host
The needs would be different.
podcast-host
In an inference world, you're mostly keeping everything within one chip.
podcast-host
It's like you wanna get you wanna dis this is where the model distillation stuff comes in.
podcast-host
You wanna get everything on one chip and do not even worry about too much of the horizontal horizontal networking.
podcast-host
And it's all about like batch size and getting stuff in, like all these people are coming in and and you this is where the CPU aspect gets more important because the you have there's more orchestration.
podcast-host
Like this job goes there, this goes there.
podcast-host
You're trying to just keep these GPUs filled.
podcast-host
What you're not doing is running these huge horizontal clusters that go that that go across like the just it's just a different structure, it's a different data center setup to be serving inference than it is to be doing training.
podcast-host
You're not.
podcast-host
Sure.
podcast-host
But and it makes
podcast-host
You're actually selling.
podcast-host
Showing 3881–3900 of 42,103 · page 195 of 2106
← Previous
Next →