Joe Carlsmith

Um,

So that's like one point on the like takeover, literal extinction thing.

I think the other thing I want to say, so I talk in the piece about this distinction between the like, well, let's at least have the AIs who are kind of minimally law abiding or something like that, right?

Like we don't have to talk about, there's this question about servitude and question about like other control over AI values.

But I think we often think it's okay to like really want people to like obey the law, to uphold basic cooperative arrangements, stuff like that.

I do, though, want to emphasize, and I think this is true of markets and true of liberalism in general, just how much these procedural norms like democracy, free speech, property rights, things that people really hold dear, including myself,

are in the actual lived substance of kind of a liberal state undergirded by all sorts of kind of virtues and dispositions and like character traits in the citizenry, right?

So like these norms are not robust to like arbitrarily vicious

Citizens so, you know, like I want there to be free speech But I think we also need to like raise our children to value truth and to know how to have real conversations And and you know, I want there to be democracy but I think we also need to raise our children to be like compassionate and decent and

And I think it's sometimes we can lose sight of that aspect.

And I think anyway, but I think like bringing that to mind now, that's not to say that should be the project of state power, right?

But I think like understanding that it's, liberalism is not this sort of like ironclad structure that you can just like hit go, you give like any citizenry and like hit go and you'll get something like flourishing or even functional, right?

There's like a bunch of other softer stuff that like makes this whole project,

I do think the even good futures will be weird.

Um, you know, I think, uh, and I want to be clear when, you know, when I talk about kind of like finding ways to, uh, ensure that kind of the integration of AIs into our society leads to good places.

Um, I'm not imagining like

I think sometimes people think that this project of wanting that and especially to the extent that that makes some deep reference to human values involves this like kind of short-sighted parochial like imposition of like our current unreflective values.

So it's just like, yeah, we're gonna have like, I don't know,

Like, I think they sort of imagine this, that we're forgetting that we too, there's a kind of reflective process and a kind of a moral progress dimension that we want to like leave room for, right?

You know, like whatever, Jefferson has this line about like, ah, you know, just as you wouldn't want to like force a man, a grown man into like a younger man's coat,