Menu
Sign In Search Podcasts Libraries Charts People & Topics Add Podcast API Blog Pricing

Mark Zuckerberg

πŸ‘€ Speaker
See mentions of this person in podcasts
6446 total appearances

Appearances Over Time

Podcast Appearances

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

But I think that there's no doubt that like energy and if you're talking about building large new power plants or large build outs and then building transmission lines that cross energy.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

other private or public land, that is just a heavily regulated thing.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

So you're talking about many years of lead time.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

So if we wanted to stand up just some massive facility to power that, I think that that is...

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

That's that's a very long term project.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Right.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

And so I don't know.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

I think that that's I think people do it.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

But I don't think that this is like something that can be quite as magical as just like, OK, you get a level of AI and you get a bunch of capital and you put it in and then like all of a sudden the models are just going to kind of like it just like I think you do hit different bottlenecks along the way.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

well i think energy is one piece yeah right um i think we would probably build out bigger clusters than we currently can if we could get the energy to do it so i think that that's um that's fundamentally money bottlenecked in the limit like if you had a trillion dollars it's time yeah right um

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Well, if you look at it in terms of, but it depends on how far the exponential curves go, right?

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Like I think a number of companies are working on, you know, right now I think, you know, like a lot of data centers are on the order of 50 megawatts or a hundred megawatts or like a big one might be 150 megawatts.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Okay.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

So you take a whole data center and you fill it up with just all the stuff that you need to do for training and you build the biggest cluster you can.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

I think that's kind of, I think a bunch of companies are running at stuff like that.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

um but then when you start getting into building a data center that's like 300 megawatts or 500 megawatts or a gigawatt i just i mean just no one has built a single gigawatt data center yet so i think it will happen right i mean this is only a matter of time but it's it's not going to be like next year right it's um i think that some of these things will take i don't know some some number of years to build out and then the question is okay well if you

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

I mean, just to, I guess, put this in perspective, I think a gigawatt, it's like around the size of like a meaningful nuclear power plant only going towards training a model.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Yeah.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

Well, that I think is a big question, is basically how that's going to work.

Dwarkesh Podcast
Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus

And I do think in the future, it seems quite possible that more of what we call training for these big models is actually...