David Duvenaud

Relatively interesting gray goo.

I'm still like, that's kind of doom.

Uh, so it's, it's like very small, right?

Then the fear is the more like, like this is what Robin Hanson fears is that we end up locking in some like, like very parochial set of values.

And, you know, maybe it's just, it's a matter of taste, but I guess to me, it looks like competition is probably going to win at the top level.

So this really reduces to like, what's the, what's the probability that there ends up being this like stable hegemon that like mostly gets values wrong.

And I'd say that that's only probably like,

like 5% or 10%.

So my PDoom, if I think that just nature flourishing or competition flourishing was valuable, would probably be only like 5% or 10%.

That's a good question.

I guess I'll say this is wide open in my mind, and I have only just started thinking about this question of at the top level, what's likely to win?

I guess I'll just say historically, there's been lots of empires and attempts to lock in values, and they've always failed.

Obviously, we're going to have much stronger coordination mechanisms, but the more levels of scaffolding we add to our civilization, also the more levels of competition there are.

Again, I have very weak intuitions here, and I feel like no one should really take my answer very seriously on this question.

Yeah, although I will say this, like the global culture, it makes it sound like this is hegemony, but there's no one steering that culture, right?

So it's not enough to become the global superpower.

You also have to control this global culture that has all sorts of its own sort of internal dynamics.

And I think just...

Controlling that is going to be a very expensive sort of thing.

It's again, it's like, why do humans get cancer?