Gavin Uberti
speaker
403 appearances
1 recordings
1 series
first heard Jun 2026
last heard 30 Jun
Gavin Uberti’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.
Appearances
There's pre-fill and there's decode.
Now, we have two key types matching both of these things.
Pre-fill is reading in a huge volume of text, and decode is then using that data to generate output tokens.
When you go out and run a pre-fill, your key job is not to go predict tokens.
You already know the text.
Your job is to go and get the model's memory, what we call as KV cache, into the right state.
Then you can go ahead and run decode with that same KV cache.
So we will often go do what we call PD disaggregation, pre-filled decode disag.
You will have one cluster of servers running these pre-fills.
You'll then transfer those model memories, those KV caches, over to the decode cluster and then go ahead and use that cluster to go generate the next tokens.
So it's sort of like loading the gun and then firing it, like if I think about it in super simple terms.
Yeah, you got it.
It's getting the model to remember the right things and then using those things to go do tasks.
So that's Pariso.
For Decode, it is all a memory game.
More memory bandwidth, you can load the model faster, load the KB cache faster, and serve more tokens per second per user.
We think people ask the wrong question here.
People often ask how much memory bandwidth is on your chip.
You should be asking how much memory bandwidth is on your full scale-up cluster.
What we were able to do is add way, way more bandwidth and a much lower latency from chip to chip.
Showing 41–60 of 403 · page 3 of 21
← Previous
Next →