David Duvenaud

And I feel like that's the biggest reason for hope, right?

So we're also going to have AIs that if they are to the extent that they're aligned to us and to the extent that they might have good ability to forecast what's actually going to be good for us, we'll be able to help us navigate this crazy world and help us choose the right means, help us tame our government.

Like everyone who's sort of worried about this issue is saying if we have good outcomes, it's probably because these AIs are really aligned to us and are really giving us a great ability to deal with these like new scary forces.

So there's a couple of reasons to think that this might not be enough though.

So one is that coordination might still be hard and it can be the case that everyone can see that something bad is likely to happen and it still can just be very hard to coordinate not to have it happen.

And again, the classic example is like world wars where everyone can see maybe that like world war one is brewing or world war three or whatever.

And all our efforts to make it not happen kind of contribute to the problem.

And I mean, maybe another simple example

of a sort of tragedy that we can all see unfolding is like companies becoming more bureaucratic over time.

It's something that it's like we all see happening.

We all know it's happened.

It's happened a million times before.

It's not really that much of a mystery, but somehow it just happens.

So I think there's, it could be the case that there's just a lot of coordination problems in the world that look like this, that even though you can all see them happening, everyone's getting good advice.

It's just so hard to address them globally.

The other reason, though, to be scared is like it's not clear that states or corporations or whatever powerful entities in the future are going to allow people to have AIs that are truly aligned to them.

And I mean, I think everyone agrees like, OK, yeah, we shouldn't allow AIs to help people build like bombs or, you know, be terrorists.

But, you know, it's also a question of like, well, what about like hate speech?

What about I want to organize like a really effective protest or like a coup or something?

What if I really don't like my government for good reasons?