Pete Huang

But even for people who deal with the real world, like plumbers, a super intelligent AI could definitely on its own figure out how to make robots that can replace plumbers.

Okay, so the consequences are massive if you can get there.

So with all of this as the backdrop around AGI and ASI and the consequences, with such dramatic outcomes, people's instincts immediately tend to be like, okay, sure, that's definitely happening.

I mean, how many times have people tried to sell others on some big lofty dream like this, right?

It's such a far-reaching sci-fi world that it can be hard to picture as reality.

Which brings us to some of the debate at hand about this set of essays that Leopold Aschenbrenner has written.

At the core again is this assumption that we're going to get another 1 million times more compute and that the models are going to keep improving at the same rate as they have been.

The honest reality is that we have no idea if that's actually the case.

Nearly every technology so far follows an S-curve.

They taper off at some point.

the more you put into it at some point, you get less and less improvement over time.

So why is this any different?

Why should we assume that we have more gas in the tank for the next five years?

Plus, there's another practical question about the data.

When you train AI models, you need the data.

And we just don't know if there's actually enough data or if we're gonna run out.

I mean, even Leopold himself acknowledges that this is a possibility.

He says that the latest AI models do already train on most of the internet.

But he goes into ways that you can improve on this, even if you do run out of internet to train on.

So for example, there's a lot of crap on the internet today, and by training on all of the internet, you're also training on all of the crap.