Joe Carlsmith

It cares about paperclips.

Maybe, maybe eventually if it's like suitably certain, it like uses, it turns itself into paperclips and who knows.

But like, um, it's still, I think a different, it's actually a somewhat different moral kind of mode with respect that, that looks to me much more like a, you know,

there's also questions like, does it, does it try to kill you and stuff like that?

But I think, I think that the, the, um, there are kind of features of the agents we're imagining other than the kind of thing that they're staring at that can matter to our sense of like sympathy, similarity, um, and, uh,

Yeah.

And I think people have different views about this.

So, so one, one possibility is that human consciousness, like the thing we care about in consciousness or sentience is super contingent and fragile.

And like most minds, most like kind of smart minds are not conscious.

Right.

It's like the thing we care about with consciousness is this hacky contingent.

It's like a product of like specific constraints, evolutionarily genetic bottlenecks, et cetera.

And that's why we have this consciousness.

And like,

you can get similar work done.

Like, so consciousness presumably does some, some sort of work for us, but you can get similar work done in a different mind in a very different way.

And you should sort of, so that's like, that's the sort of consciousness that's fragile view.

Right.

And I think there's a different view, which is like, no consciousness is, is, um,

something that's quite structural.