David Duvenaud

And I'm kind of happy with that.

But again...

I feel like we have to screen off the roles where we think that just whatever happens, like whoever lives is going to be happy and that's fine because then it doesn't really matter what we do.

And I would rather worry about the roles where like it actually matters what we do.

Oh, absolutely.

I mean, yes.

I think I've mentioned it in passing, but that absolutely, I think, is a big part of what we can do now.

And it's maybe not even all that under-incentivized, though.

And yeah, AIs are just going to...

Even things like crypto or whatever, there's just all these experimental things that will let us have much more powerful coordination mechanisms than we have in the future.

And I think that's a big part of avoiding doom, is developing those before we are disempowered in more serious ways.

But all of this is dual use because

It's harder for humans to participate in these coordination mechanisms than AIs, so it's not really clear which way speeding this up cuts.

But in general, I'd probably be in favor of developing it faster.

Well, the way it backfires is like, you know, think of like people are always trying to build like decentralized autonomous organizations.

And it's like kind of doesn't work for like a number of reasons.

But that's the sort of thing where people could accidentally create these sort of like self-replicating like beings or polities that get to live in some sort of margins and be like really hard for us to shut down.

And I think people are going to be constantly like seeding the world with these attempts at like self-replication.

sustaining machine life and civilization, just for various reasons.

Like I'm saying, I want to be on the right side of history.