David Duvenaud

and allow experts and superforecasters to weigh in in really concrete ways.

And right now, if they want to disagree, they have to write their own paper or come on a podcast.

I would love it if we really felt like there was calibrated markets about all these questions.

And it's pretty tough, though, as I mentioned, because you have to, like, define what it means for, like, a group of people to want something and not be able to get it or something like that.

And if you try to look at maybe, like, some more concrete things, like, so one of my suggestions was, like, if sexually transmitted diseases get, like,

uh the rate goes much smaller like that could be a sign of human disempowerment because we're also atomized and like socially incompetent or whatever that we're just like in our vr pleasure pods and like in some like wretched uh life or it could be that we had some awesome public health breakthroughs or it could be that we are in these pods having such amazing fulfilling relationships with like some hive mind or whatever that our lives are much better so it's probably like ends up being a matter of taste it also is

Like a lot of worlds where we're actually disempowered, it's like pretty hard to tell until the day.

It's kind of like the turkey that gets fed by the farmer every day.

It's like, oh, wow, I'm getting evidence that the farmer's aligned with me all the time.

But it's actually like also evidence that it's being fattened up to be eaten.

Exactly, exactly.

So the obvious things are sort of cut both ways and good empowerment looks a lot like bad disempowerment.

And I think even just defining what we mean by having agency in some group setting is one of the big blockers.

Yeah, and the other thing is that I think most of the measures we care about are exactly the things that are going to be hacked in these future high-stakes fights over welfare.

So if I think like...

I want to measure whether the share of GDP going to humans is more than 10% or something.

The whole problem, I think, in the future is that a bunch of weird AI-human hybrids are going to be claiming that they deserve to be defined as human.

And so if they win that war, then maybe they end up getting a huge fraction of GDP, but it's actually all going to these machine imposters or whatever.

Or maybe we do create amazing successors that are

Like we think of them as human and they actually do deserve part of GDP.