Daniel Kokotajlo

Your ability to make sense of the environment is just so little loaded on intelligence and so much more loaded on your ability to experiment and your ability to communicate with other people and pass down knowledge over time.

Part of me was just like, I was doing that interview, I'm like, maybe ESI is fake, maybe just like... Let's hope.

That's right.

So what we're asking is to just radically boost up the technological maturity of civilization within the matter of years.

Or get us to the Dyson Spheres in the matter of years.

Rather than, yes, maybe causing a 10x-ing of the research.

But I think human civilization would have taken centuries to get to the Dyson Spheres.

I agree.

We're on a totally different topic here of do you get the Dyson sphere?

There's one world where it's crazy, but it's still boring in the sense of

You know, the economy is growing much faster, but it would be like what the Industrial Revolution would look like to somebody in the year 1000.

And that one is one where, you know, you're still trying different things.

There's failure and success and experimentation.

And then there's another where it's like the thing has happened, and now you send the probe out, and then you look out at the night sky six months later, and you see something occluding the sun.

In your story, you have basically two different scenarios after some point.

So yeah, what is a sort of crucial turning point and what happens in these two scenarios?

So in the world where they're getting deployed through the economy, but they are misaligned.

And you, you're, you know, people in charge, at least at this moment, think that they are in a good position with regard to misalignment.

It just seems with even smart humans, they get caught in weird ways because they don't have logical omniscience.

They don't realize the consequences of the way they did something which just obviously gave them away.