Gavin Uberti
speaker
403 appearances
1 recordings
1 series
first heard Jun 2026
last heard 30 Jun
Gavin Uberti’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.
Appearances
to our interconnects.
It allows us to be able to go serve models at this much higher speed because you can go use the SRAM and the HBM from the full scale up cluster as a single pool.
And that's our second key technical bet, what we call cluster scale memory.
And on GPUs today, the cluster memory bandwidth is often very badly utilized because the time to go hop from one GPU to another is extremely long.
For example, on Blackwell chips, it can be about 4,000 nanoseconds to go point to point.
And that means that if you go ahead and go to an 8x TP setup, you will get way, way less than an 8x improvement in your tokens per second per user.
And what we did is built our own totally custom interconnect stack.
When you took
Everything above the second layer Ethernet built at full custom.
And we can go out and do far for better latencies and bandwidths this way too.
We can go ahead and cut this by more than a factor of 5x.
And that allows us to then use the memory of other chips much more effectively.
As you scale the world size, your time per token goes down proportionally.
And I think that if you look at most other goods, like the iPhone, for example, they've gotten into this economies of scale, where as a result, more money does not really buy a better iPhone.
That if you're a billionaire or if you're just the average American, you buy the same phone.
And tokens aren't like that yet.
We're still in the very early days where the general purpose system, relatively small one, is kind of hand crafting these tokens like they made screws back in the late Renaissance.
And I wanted to live in a world where you have the same economies of scale for token making that you do for making say iPhones or cars or anything else.
I think that is one of the huge unlocks that allows a huge group of people to go use the best quality models.
economies of scale have all made capitalism very, I don't know, fair.
Showing 61–80 of 403 · page 4 of 21
← Previous
Next →