Gregory Carr
speaker
2,075 appearances
6 recordings
6 series
first heard Dec 2025
last heard 7 Jul
Gregory Carr’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 6 in all, peaking in Jun 2026 with 4.
Appearances
Are you interested in putting in a Mantrap?
This is 100% true, but everything that cuts against training works for inference.
Respect.
Well, latency is super important.
And I do think, you know, a distributed inference, distributed inference clouds are coming.
And to riff on like what Tomas said and all of this, like one, I mean, there is actually a startup.
that is trying to put four gpu units with kind of a battery on people's houses and give them a discount on their power and then you can do inference for that neighborhood you know from those four gpus and it's like lock sealed so nobody can get in but there's there's another dynamic that i think we should talk about with all of this and you can play into the mega pods and
you know, other people are kind of working on data centers.
You know, Crusoe is working on, you know, modularly assembling data centers, you know, kind of like a data center and think of it as like an 18 wheeler.
What do you call those things?
The 18 wheeler, you know, shipping container, whatever it is.
Um,
But that is the disaggregation of inference into pre-fill and decode.
When a model is answering your question, it's doing two things.
The pre-fill part is understanding the question and its answer thus far.
And think of that as the more you can remember, the bigger your memory capacity, literally the more words you can remember, the better.
Decode is the process of generating the next token.
And that is a memory bandwidth bound problem.
And think of it as the faster you can speak, the better.
And these two types of inference are increasingly being disaggregated.
Showing 461–480 of 2,075 · page 24 of 104
← Previous
Next →