Pete Huang

At the core again is this assumption that we're going to get another 1 million times more compute and that the models are going to keep improving at the same rate as they have been.

The honest reality is that we have no idea if that's actually the case.

Nearly every technology so far follows an S-curve.

They taper off at some point.

the more you put into it at some point, you get less and less improvement over time.

So why is this any different?

Why should we assume that we have more gas in the tank for the next five years?

Plus, there's another practical question about the data.

When you train AI models, you need the data.

And we just don't know if there's actually enough data or if we're gonna run out.

I mean, even Leopold himself acknowledges that this is a possibility.

He says that the latest AI models do already train on most of the internet.

But he goes into ways that you can improve on this, even if you do run out of internet to train on.

So for example, there's a lot of crap on the internet today, and by training on all of the internet, you're also training on all of the crap.

What if you didn't?

What if you trained on the entirety of the internet, but only the section that is high quality and not crap?

That might make the models better.

So there's plenty of debate about whether or not we actually are going to get to AGI on the pathway that we're on.

And by the way, we haven't even talked about what to do about all this.

That's a whole different can of worms.