David Duvenaud

It's not really clear how a single company can deal with it.

And they end up doing this thing where the RSP addresses or the...

whatever steeply commitments address these like very acute catastrophic risks.

And then there's just a whole bunch of slower sort of more systemic ways that things goes wrong that it's not clear what, like if everyone's getting like AI girlfriends and boyfriends, like how are they supposed to address that?

Right.

It's not, it's clearly just not within their scope.

Sure.

I mean, I think the best example of this is just all of the AI lab leaders being convinced by this story that like, Oh, this is an existential risk and question mark, question mark, question mark.

And now I need to join the race and build my own AI lab just to even matter.

Right.

And also, you know, I think a lot of people ask like, okay, in the, in the future, why are the oligarchs not coordinating?

And it's like, well, in particular, like the reason that XAI and, uh,

OpenAI and Anthropic all exist is because these particular oligarchs actually exactly failed to get along on exactly this topic.

Yeah.

Yeah, I mean, well, I don't know what their relationships are in much detail, but I guess I'll just say, like, these are the exact people that we were hoping will coordinate on exactly this issue, and they already exactly are failing to do so right now.

Yeah, I mean, we've thought a lot about this.

And like my co-author, David Kruger, wears t-shirts at like New Reps that say like, just don't build AGI and gives out stickers and stuff like that.

And I think...

People under-discuss this possibility because it's not very fun.

It's very wealth-destroying.