David Duvenaud

I mean, yes.

I think I've mentioned it in passing, but that absolutely, I think, is a big part of what we can do now.

And it's maybe not even all that under-incentivized, though.

And yeah, AIs are just going to...

Even things like crypto or whatever, there's just all these experimental things that will let us have much more powerful coordination mechanisms than we have in the future.

And I think that's a big part of avoiding doom, is developing those before we are disempowered in more serious ways.

But all of this is dual use because

It's harder for humans to participate in these coordination mechanisms than AIs, so it's not really clear which way speeding this up cuts.

But in general, I'd probably be in favor of developing it faster.

Well, the way it backfires is like, you know, think of like people are always trying to build like decentralized autonomous organizations.

And it's like kind of doesn't work for like a number of reasons.

But that's the sort of thing where people could accidentally create these sort of like self-replicating like beings or polities that get to live in some sort of margins and be like really hard for us to shut down.

And I think people are going to be constantly like seeding the world with these attempts at like self-replication.

sustaining machine life and civilization, just for various reasons.

Like I'm saying, I want to be on the right side of history.

I think it's like every month for the foreseeable future, there's going to be at least one attempt where someone's like, I gave this AI a Bitcoin wallet and asked it to go make money or something like that and try to start these self-sustaining little societies.

And to the extent that those AIs are able to just coordinate amongst themselves, that's another source of danger.

Yeah, so I'm not doing much of this.

This is like Deir, Jan, some other people.

The idea is to try to operationalize some concrete questions that help us at least settle the questions of, you know, do we think that this kind of loss of control that I'm talking about is going to happen?