Jeff Dean

speaker

343 appearances 1 recordings 1 series first heard Aug 2026 last heard 1 Aug

Jeff Dean’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Aug 2026 with 1.

Appearances

newest first · ▶ plays the moment

Y Combinator Startup Podcast · Jeff Dean: The 1% Rule for Building in AI · 1 Aug 2026 podcast

Yeah, I mean, I think the example you raised of a thousand X difference in bringing moving data versus actually computing on it in terms of energy is a pretty significant one.

And it shapes a lot of aspects of what we do in machine learning.

because if you didn't have that thousand X difference, then, you know, you wouldn't have to do batching, but you have to do batching of, you know, many examples or maybe many tokens at once in order to amortize that data movement so that you can, you know, not pay a thousand X slowdown, but pay a thousand X divided by batch size energy cost.

And for really low latency, batching is not really very good.

So I think these kinds of things and the energy behind various decisions in the computer hardware we use really affects a lot of decisions we make in building higher level systems.

Yeah, I mean, you have to assemble batches to get better efficiency in your hardware.

Ideally, you might do batch size one training, but it's not as good in terms of efficiency.

So people use pretty large batches these days.

Yeah, I've been thinking more about inference, actually.

So I think inference is a pretty interesting problem because you do want very low latency.

Training, you don't necessarily need incredibly low latency.

And I think there's a lot of room for specializing hardware more for inference than we are today.

I mean, just trying to minimize data movement, trying to think about incredibly low precision operations, and maybe not supporting lots and lots of different kinds of precisions.

If you feel like you have a good answer for what kinds of precision you need, maybe just build that into the hardware and not much else.

Yeah.

I mean, if you truly understand the data, you should be able to compress it really well.

Yeah, yeah, I would say.

Good work by my colleagues.

Yeah, I mean, I think the model is really only one piece of what you're trying to do, which is build an overall system that can solve really interesting problems.

And that involves a model that knows how to use various tools.

Showing 61–80 of 343 · page 4 of 18