Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Jeremiah

๐Ÿ‘ค Speaker
1129 total appearances

Appearances Over Time

Podcast Appearances

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Building data centers isn't illegal, is it?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Oh, sorry, I'm not using in Minecraft as a euphemism for it being a crime.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

We're literally building the data centers in Minecraft.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Did you hear about the guy who made a working language model in Minecraft using redstone circuits?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Pretty amazing, isn't it?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

His version is barely GPT-2 level, but there's no reason we can't scale that up.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Once we create full-sized data centers in Minecraft, everyone will want to do their training runs there.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

What do you mean, why?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Real world data centers cost billions of dollars, raise electricity prices, waste, she briefly scans the room to confirm Andy Masley isn't around, then continues, water, and they're getting increasingly politically unpopular and hard to build.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

We can short-circuit all of that by putting the data centers in Minecraft instead.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

But you have to have the Minecraft world being simulated by real computers, right?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

So don't you still need the data center in order to play the Minecraft?

Astral Codex Ten Podcast
SOTA On Bay Area House Party

Oh, I'm sure you need some computer, but it's a question of leverage.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

One high-end gaming computer playing Minecraft can include a whole world with continents, mountain ranges, forests, and oceans.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

You can fit thousands of data centers in that world.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

So with even one real-world computer, you've saved billions on chips and construction costs.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

You take a moment to consider how best to explain this.

Astral Codex Ten Podcast
SOTA On Bay Area House Party

So, um, every computation has to be done somewhere, right?