David Duvenaud

You're not part of the cool kids club if you're not building.

And I'm one of these people that just loves to build.

We mentioned this project on LLM forecasting, for instance.

But anyway, I think lately we've all agreed that we should make a habit of saying, if we could all coordinate to delay building much more powerful superhuman systems, that would be a good thing.

And I think the realistic way this would go is that this would end up...

consolidating most of the private efforts into some government programs, which, again, I fear the government, and I think this is very scary for a number of reasons, but it's, I think, probably still barely on the margin of positive development.

So I don't think it's very feasible, but I want to mention in the Kickstarter spirit, if everyone else agrees, I would also think this is a good effort to support.

Yeah, exactly.

And like basically harder to recover from if it ends up being captured by someone who like doesn't care about like various forms of safety or like the public good or something like that.

Yeah.

So, I mean, definitely the fear though is that we end up with some sort of warning shot that doesn't

uh sort of unites people it actually actually polarizes people and so like yasha is somebody who's um like has discussed this and mentioned that there's a lot of disasters that look like social disasters where afterwards everyone can agree that something bad happened but they totally disagree on what it was and so maybe covet is a perfect example where you know half of people think oh man code was such a disaster because the public health people didn't go far enough and they didn't like control the

virus and people weren't on board with all the public health measures and stuff like that.

And half of people think COVID was a disaster because public health totally overreacted and the government had all these authoritarian pointless crackdowns that were so destructive.

And so we can all agree there's a warning shot, but half of us think the answer is dismantle public health and half of us think that the answer is strengthen public health.

And I think you can easily imagine a lot of

warning shots where there's like, I don't know, an A.I.

influencer that makes like some sort of destructive cult and gets shut down by the Internet.

And then half of people think that like Mecca influencer didn't go far enough and half of them thinking Mecca influencer went too far.

And so half the people think that like we need to protect future A.I.