Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
So you're like typing your query and it's kind of like honing in on, and you know, it's like, okay, here, you know, show me a picture of a cow, okay, in a field with mountains in the background.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
It's just like everything's popular.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
Eating macadamia nuts, drinking beer.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
And like, it's just like, it's updating the image in real time.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
It's pretty wild.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
I think people are going to enjoy that.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
So, yeah, so that I think is that's what most people are going to see in the world.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
Right.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
We're rolling it out, you know, not everywhere, but we're starting in a handful of countries and we'll do more over the coming weeks and months.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
So that's that I think is going to be a pretty big deal.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
And I'm really excited to get that in people's hands.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
it's a big step forward for Meta AI.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
But I think, you know, if you want to get under the hood a bit, the Llama 3 stuff is obviously the most technically interesting.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
So, you know, we're basically, for the first version, we're training three versions, you know, an 8 billion and a 70 billion, which we're releasing today, and a 405 billion dense model, which is still training.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
So we're not releasing that today.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
But
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
The 8 and 70, I'm pretty excited about how they turned out.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
They're leading for their scale.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
We'll release a blog post with all the benchmarks so people can check it out themselves.
Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus
Obviously, it's open source so people get a chance to play with it.