David Duvenaud

So that's that's one weak effect that used to keep culture roughly in line with human flourishing that's going away.

The other thing that's going to start to happen is machines producing culture, right?

Like once machines sort of like become more like agents with their own independent beliefs and sort of point of view on the world, talking more directly to each other, then this is like a new thing in history, which is there's like a new vessel of cultural sort of memes and creation and just like norms that can be operating sort of almost mostly independently from humans.

And it could end up developing in a kind of anti-human way.

And then the third thing is just we're going to be spending so much time talking to machines that this is like a new way that culture is going to transmit.

That's just going to look very different than how we transmit culture today.

Yeah, and I think people are really recognizing that the beliefs of AIs or the constitutions of AIs are a key front in cultural battlegrounds.

It's just like people used to fight over Wikipedia to try to set the narrative.

And I think now if you want to set the narrative on some controversial topic, if you can really control how ChatGPT frames it, that's going to set what the default cultural answer is.

In a sense, this is business as usual.

People already notice that

economics or cultural forces or like geopolitical forces end up pushing us towards outcomes that I think no one would endorse.

And we argue that the development and proliferation of smarter than human AI is going to make those forces even stronger and remove some of the safeguards that tend to keep our civilization serving human interests in the long run.

Maybe a simple example is just clickbait short-form video content or something like that, where maybe the consumers realize it's not the best use of their time.

They kind of regret spending a lot of time on it.

And the producers also know what they're doing, and they're making clickbait, and they know it.

But if they want to make their educational long-form content

content like such as this, then they get punished by not having as many views.

And now each of these content creators having their own aligned AI doesn't solve this like global problem of the market just incentivizing for this sort of not very helpful to humans cultural content.

Yeah, so that's a good example of how more optimization pressure in our current civilizational incentives is going to probably be a bad thing in a lot of ways.