David Duvenaud

It just seems like we'll be in a very vulnerable position.

Yeah.

So one of them is just humans not being that close to the action or knowing what's going on.

And maybe a concrete example today is if I'm an average Joe and I think like actually AI is going to be the next big thing and I really want to invest in it.

Most of the big AI companies are still private equity, right?

Like Anthropic and XAI.

And so there's actually no direct way for me to even invest in them.

And of course, like bigger investors can.

But this is a good example of like no one's making this happen.

It just happens that our institutions...

marginalize almost everyone from participating in like one of the biggest wealth creation events in human history.

So again, AI advice, helping you avoid these situations is like maybe the thing that keeps us safe.

But again, there's just going to be incentives for the people who are actually creating the, or like the, not people, the AIs that are creating the wealth just naturally are going to form these sort of like local bubbles of enrichment, which will be hard for the larger, like massive unproductive humanity to participate in.

Yeah.

And that's exactly what I was thinking of when I was referring to these like sort of bubbles of wealth creation is like also all these new beings.

This question of like legal personhood, I also think it's very unstable to have the sort of like very productive sort of cutting edge, like whatever is most like memetically fit beings just forever not being able to de facto grab the mantle of power one way or another.

The problem is that this is all very fuzzy and far into the future, so any particular scenario sounds a bit far-fetched, especially without all the other parts of the scaffolding set up to realize why this locally will seem inevitable, I think.

Sure, I guess maybe one way to think about it is what are the forces that are acting towards equality that are going to stop operating?

And the main one is just the value of labor.

So right now, sort of everybody for most of their lives has some valuable labor that they can trade.