Dwarkesh Patel

speaker

19,288 appearances 62 recordings 3 series first heard Feb 2024 last heard 17 Sep

Dwarkesh Patel’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 27 in all, peaking in Jun 2026 with 6.

Appearances

newest first · ▶ plays the moment

Dwarkesh Podcast · Reiner Pope – The math behind how LLMs are trained and served · 29 Apr 2026 podcast

If you want an LLM that you can run locally on your laptop or even your phone, you should check out Gemma 4.

When was GPT-4 released again?

It was 2022 or 2023?

Three.

And it was rumored to be over 1 trillion parameters.

And it seems like only now and within the last six months have models been getting released that are significantly more parameters than a model released three years ago.

Yeah.

When supposedly there should have been this scaling in the meantime.

Is the reason that we were just waiting for RACs

with enough memory to hold a five trillion parameter model along with its kv cash for enough you know users for a full um for a lot of sequences or rl if you're doing rl kind of a similar consideration of actually holding the kv cash for all the the uh the the batch of problems you're trying to solve um so if you look at like hopper you had eight hoppers and i think the

That's 640 gigabytes as of 2022.

Yeah.

With Blackwell finally, which was deployed, what, 2020?

Very recently.

I mean, last year.

Last year?

Yeah.

You finally have a scale up with on the order of like 10, 20 terabytes, which is enough for like a 5T model plus KB cache.

And that also explains why Gemini seemed to be ahead.

Gemini 2.5 was a successful, or it just seems like Gemini has that successful pre-train for longer than some of the other apps.

Showing 4501–4520 of 19,288 · page 226 of 965