Dwarkesh Kheterpal

All right, Mark, thanks for coming on the podcast again.

Yeah, happy to do it.

Good to see you.

You too.

Last time you were here, you had launched Llama 3.

Yeah.

Now you've launched Llama 4.

Well, the first version.

That's right.

What's new?

What's exciting?

What's changed?

Oh, well, I mean, the whole field's so dynamic.

I'm interested to hear more about it.

There's this impression that,

that the gap between the best closed source and the best open source models has increased over the last year, where I know the full family of Lama 4 models isn't out yet, but Lama 4 Maverick is 35 on Chatbot Arena, and on a bunch of major benchmarks, it seems like 04 Mini or Gemini 2.5 Flash are beating Maverick, which is in the same class.

What do you make of that impression?

Yeah, well, okay, there's a few things.

Do you feel like there is some benchmark which captures what you see as a North Star of value to the user, which can be sort of objectively measured between the different models?

And you're like, I need Lama 4 to come out on top on this.