Dwarkesh Patel

And I think this analogy actually has remarkable reach.

Just because fossil fuels are not a renewable resource does not mean that our civilization ended up on a dead-end track by using them.

In fact, they were absolutely crucial.

You simply couldn't have transitioned from the water wheels of 1800 to solar panels and fusion power plants.

We had to use this cheap, convenient, and plentiful intermediary to get to the next step.

AlphaGo, which was conditioned on human games, and AlphaZero, which was bootstrapped from scratch, were both superhuman Go players.

Now, of course, AlphaZero was better.

So you can ask the question, will we or will the first AGIs eventually come up with a general learning technique that requires no initialization of knowledge and that just bootstraps itself from the very start?

And will it outperform the very best AIs that have been trained up to that date?

I think the answer to both these questions is probably yes.

But does this mean that imitation learning must not play any role whatsoever in developing the first AGI or even the first ASI?

No.

AlphaGo is still superhuman despite being initially shepherded by human player data.

The human data isn't necessarily actively detrimental.

It's just that at enough scale, it isn't significantly helpful.

AlphaZero also uses much more compute than AlphaGo.

The accumulation of knowledge over tens of thousands of years has clearly been essential to humanity's success.

In any field of knowledge, thousands and probably actually millions of previous people were involved in building up our understanding and passing it on to the next generation.

We obviously didn't invent the language we speak, nor the legal system we use.

Also, even most of the technologies in our phone were not directly invented by the people who are alive today.