David Duvenaud

And it's not clear in an absolute sense how hard it even is.

I think it's very, very hard.

But the stakes also haven't been very high so far.

So it's one of these, like, you always have to ask, like, why has no one else...

looked at this in depth before.

And certainly people have, I'm not claiming we're the first ones, but it still seems massively understudied to me.

And I think part of the reason is that the stakes just haven't been that high so far.

Yeah.

And basically, if you think Malthusian dynamics look like a bad end, no matter what, it's kind of unclear what would even make you happy about the distant future.

So my current way of thinking about it is that we are going to be optimizing some value function just because we're these ambitious beings who have wants and agency.

If we end up having to optimize sort of for growth, then we will lose a lot of what we value.

The good end looks like we managed to control our own sort of fitness function that we end up spending the rest of our days optimizing.

And it's going to take a lot of thought to get that right in such a way that doesn't destroy almost all value by our current standards.

Well, I mean, OK, so let me try to steal a man the position.

So one thing is like, again, liberalism is unintuitively good just for everybody involved, especially even the people who are like poor or like disenfranchised.

They still get like better welfare and stuff.

So that's one reason why, like historically, people were afraid of like the Industrial Revolution, but it ended up being good.

People are making the same argument now and we're making the case that it's actually importantly different now, which I think is like.

I think people are right to be skeptical, right?

Because everyone's always saying, don't automate my jobs, it's going to ruin my life.