Scott Alexander

Instead, they sort of like negotiate and come to some sort of deal where they basically share power.

And like there is this oversight committee that has some members appointed by the president and then also like the CEO and his people.

And like that committee votes on high level questions like what goals should we put into the super intelligences?

Well, to be clear, we are uncertain about this.

But in our story, we depict this sort of intense wake-up happening over the course of 2027, mostly concurrently with the AI companies automating all of their R&D internally and having these fully autonomous agents that are like amazing autonomous hackers and stuff like that, but then also just like actually doing all the research.

And part of why we think this wake-up happens is because the company deliberately decides to wake up

the president.

And this is a, you could imagine running the scenario without not happening.

You can imagine the company is trying to sort of keep the president in the dark.

I do think that they could do that.

I think that if they like didn't want the president to wake up to what's going on, they might be able to achieve that.

Strategically, though, that would be quite risky for them because if they keep the president in the dark about the fact that they're building superintelligence and that they've actually completely automated their R&D and it's getting superhuman across the board, and then if the president finds out anyway somehow, perhaps because of a whistleblower, he might be very upset at them, and he might crack down really hard and just actually execute on all the threats and nationalize them and blah, blah, blah, blah, blah, blah, blah.

They kind of want him on their side, and to get him on their side, they have to make sure he's not surprised by...

any of these crazy developments.

And also, if they do get him on their side, they might be able to actually go faster.

They might be able to get a lot of red tape waved and stuff like that.

And so we made the guess that early in 2027, the company would basically be like, we are going to deliberately wake up the president and scare the president with all of these demos of crazy stuff that could happen and then use that to lobby the president to help us go faster and to cut red tape and to maybe slow down our competitors a little bit and so forth.

But perhaps this is a good point to mention.

This is an epistemic project.

We are trying to predict the future as best as we can, even though we're not going to succeed fully.