David Duvenaud

That's just going to look very different than how we transmit culture today.

Yeah, and I think people are really recognizing that the beliefs of AIs or the constitutions of AIs are a key front in cultural battlegrounds.

It's just like people used to fight over Wikipedia to try to set the narrative.

And I think now if you want to set the narrative on some controversial topic, if you can really control how ChatGPT frames it, that's going to set what the default cultural answer is.

In a sense, this is business as usual.

People already notice that

economics or cultural forces or like geopolitical forces end up pushing us towards outcomes that I think no one would endorse.

And we argue that the development and proliferation of smarter than human AI is going to make those forces even stronger and remove some of the safeguards that tend to keep our civilization serving human interests in the long run.

Maybe a simple example is just clickbait short-form video content or something like that, where maybe the consumers realize it's not the best use of their time.

They kind of regret spending a lot of time on it.

And the producers also know what they're doing, and they're making clickbait, and they know it.

But if they want to make their educational long-form content

content like such as this, then they get punished by not having as many views.

And now each of these content creators having their own aligned AI doesn't solve this like global problem of the market just incentivizing for this sort of not very helpful to humans cultural content.

Yeah, so that's a good example of how more optimization pressure in our current civilizational incentives is going to probably be a bad thing in a lot of ways.

There's also going to be a better ability for us to sort of see what's happening and coordinate thanks to these aligned AIs.

So these two forces are going to be working against each other.

And one of our central claims is it's not at all clear which one is going to dominate in the long run.

Yeah.

So that's a great question.