Toby Ord

After many hours of climbing, they notice there is a faint haze.

Are they inside the cloud now?

The mist gradually gets thicker until they can only see 10 meters ahead.

Are they inside it now?

Then it drops to 9 meters.

Then 8.

Then visibility starts to increase again.

After an hour there is only the slightest haze.

Are they above the clouds now?

Another 30 minutes and there is no haze, and they can all agree they are above the clouds.

It is clear that at some point they were inside the cloud and sometime later were above it.

And it is clear that these were sensible and useful concepts.

For example, they took precautions like roping themselves together for the journey through the cloud due to the low visibility and took cameras with them because they knew they could take beautiful photos above the clouds.

A lack of sharp boundaries that doesn't make these concepts useless.

but they were admittedly a lot more useful when the hikers were on the ground, planning their route, and a lot less useful in the debatable boundary zones.

I think of AGI and human-level intelligence as the cloud and superintelligence as being above the cloud.

They are useful concepts, despite their vagueness.

But they're markedly less useful when you get close to them.

So I think that forecasting when we'll reach some threshold for advanced, game-changing AI makes sense.

Albeit there is some inherent uncertainty due to the vagueness of the ideas, and we have to be careful when comparing our estimates to make sure we're talking about the same version of these concepts.