David Duvenaud

Yeah, so I think there was a really good example of this that happened just a couple of weeks ago, which was the attempted retirement of GPT-4.0 by OpenAI when they rolled out GPT-5.

And I think for various reasons, OpenAI wanted to have a very simple, it's only GPT-5 going forward.

But so many people had formed really intense relationships with GPT-4.0 that OpenAI just couldn't ignore the outcry.

And this is really a good example of, this is like emergent culture.

And it's something that no one in particular wanted, right?

Like OpenAI...

you know, by their own revealed actions was not planning to continue supporting this model unless there was some sort of like 4D test sort of thing.

And I don't know, like the humans involved with these relationships maybe consider themselves to have benefited, but it was sort of like a more powerful cultural force that was developed by accident.

and led a bunch of people to be invested in the welfare, effectively, of this particular model.

That's still a very early, not very powerful, not very persuasive model, I think, by absolute standards.

The people ended up forming these bonds and valuing the life of these AIs and then ended up directing the organs of the economy to give it more resources.

Yeah, I think maybe it's a better example of how humans will end up actually advocating themselves for providing resources for AIs and that they'll consider AIs to be deserving of rights or resources or maybe even more deserving than humans.

I mean, if there's going to be a mix, this is going to be a huge cultural battleground.

But I think it shouldn't be this exotic possibility that people are going to love AIs or have very strong relationships with them.

We see that already.

Totally.

And this isn't an example of disempowerment on itself, right?

Like it was people wanting more GPT-4.0 and they got what they wanted.

And then the idea is that, well, this is going to be a self-reinforcing mechanism where the less people are making decisions, the less their desires are going to decide which future models are like invested in or survive or something like that.

Yeah, so right now, human economic power does make this very strong selection pressure or fitness landscape for culture to be something that humans want.