David Duvenaud

Sure, sure.

So I think a lot of people might say, hey, you don't have much moral imagination.

Why are you insisting on like these human well-being or human desires when we know that in principle, there's definitely going to be like more morally deserving things in the future or something like that?

And my basic answer is, well,

In some sense, we decide what is morally deserving, and it would be really surprising if for those beings to exist in the best possible world, we all had to die and have some terrible time.

So we basically don't have to decide between these different views, and we can just say, hey, let's try to make sure that something like existing humans get to decide roughly what's done with the rest of the universe or the future or whatever, if that involves having these sort of Amish-style...

leave earth as a nature preserve, like whatever it is, let's just like let ourselves decide and not let it be up to some sort of like race to the bottom, Milwaukee and like dynamics where we end up choosing something that no one endorses.

They're basically the same.

I think it would be really weird if we somehow accidentally killed and disempowered existing humans and ended up building some sort of future that those humans would otherwise really endorse.

I think the default is there's some sort of like locust-like beings that just like growth for growth's sake.

And that's sort of the default thing that all evolutionary pressures select for.

And, you know...

Maybe those beings are pretty cool.

I don't know.

And if they are, then it doesn't really matter what we do.

So we don't really have to worry about that scenario.

But the scenario where they're just kind of like this gray goo that we think is a big waste, that's what we need to avoid.

And if you and I are on Earth flourishing for a long time, and the state and all our civilization apparatus is acting in our interest, and we decide, hey, actually, it would be amazing to create this type of future, then just as part of serving our interests, we would end up creating that amazing future.

But that doesn't mean that I don't take a lot of morality very seriously.

I'm just trying to say the sort of fact of the matter is determined by what's in our heads and then also whatever conditions that imposes on the world being good.