Daniel Kokotajlo

It would have to be for millions of them in parallel to be rapidly speeding up AI research.

And there just is this, you know, there's this quote that Napoleon, one Napoleon is worth 40,000 soldiers, was commonly a thing that was said when he was fighting.

But 10 Napoleons is not 400,000 soldiers, right?

So why think that these million...

AI researchers are netting you something that looks like an intelligence explosion.

Is there some intuition pump from history where there's been...

Some output, and because of some really weird constraints, production of it has been rapidly skewed along one input, but not all the inputs that have been historically relevant, and you still get breakneck progress.

So maybe the reason that this sounds less plausible to me than the 25x number implies is that when I think about concretely what that would look like, where you have these AIs and we know that there's a gap in data efficiency between human brains and these AIs.

And so somehow there's a lot of them thinking and they think really hard and they figure out how to define a new architecture that is like AI.

the human brain or has the advantages of the human brain.

And I guess they can still do experiments, but not that many.

Part of me just wonders like, okay, what if you just need an entirely different kind of data source that's not like pre-training for that, but they have to go out in the real world to get that?

Or maybe they just need to...

it needs to be actively, it needs to be an online learning policy where they need to be actively deployed in the world for them to learn in this way and you show your bottleneck on how fast they can be getting real-world data.

But even in that scenario alone, I can imagine bottlenecks like, oh, you had a benchmark and it got reward hacked for what constitutes AI R&D because you obviously can't have, like, what is the... Maybe you would, but...

is it as good as a human brain?

It's just such an ambiguous thing.

Right now we have benchmarks that get reward hacked, right?

What about the skepticism that...

Look, what you're suggesting with this hyper-efficient hive mind of AI researchers, why no human bureaucracy has just out of the gate worked super efficiently, especially one where they don't have experience working together.