Jeff Dean

So people are going to want to spend some fraction of world GDP on this thing.

The world GDP is almost certainly going to go way, way up to orders of magnitude higher than it is today due to the fact

that we have all of these artificial engineers working on improving things.

Probably we will have solved unlimited energy and carbon issues by that point.

So we should be able to have lots of energy.

We should be able to have millions to billions of robots building us data centers.

Like, let's see, the sun is, what, 10 to the 26th watts or something like that.

I mean, I'm guessing that the amount of compute being used for AI to help each person will be astronomical.

That would be so nice.

You could just glue models together or rip out pieces of models and shove them into other, like Dr. Frankenstein-y kind of thing.

Or you just attach a fire hose and you suck all the information out of this model and shove it into another model.

There is, I mean, the countervailing interest there is sort of science in terms of like, OK, we're still in the period of rapid progress.

So if you want to do sort of controlled experiments

OK, I want to compare this thing to that thing because that is helping us figure out, OK, what do you want to build?

So in that interest, it's often best to just start from scratch so you can compare one complete training run to another complete training run sort of at the practical level.

because it kind of helps us figure out what to build in the future.

And it's less exciting, but does lead to rapid progress.

Actually, that could lead to faster research progress, right?

You've got some system and you do something to improve it.

And if that thing you're doing to improve it is...