Joe Carlsmith

It's much more defined by functional roles like self-awareness, a concept of yourself, maybe higher order thinking, stuff that you really expect in many sophisticated minds.

And in that case, okay, well now actually consciousness isn't as fragile as you might have thought, right?

Now actually like lots of beings, lots of minds are conscious and you might expect at the least that you're gonna get like conscious super intelligence.

They might not be optimizing for creating tons of consciousness, but you might expect consciousness by default.

And then we can ask similar questions about something like valence or pleasure or like the kind of character of the consciousness, right?

So there's, you can have a kind of cold, indifferent consciousness that has no like human or no like emotional warmth, no like pleasure or pain.

I think that can still be, Dave Chalmers has papers about like Vulcans and he talks about they still have moral patient hood.

I think that's very plausible, but I do think it's like,

Um, an additional thing you could get for free or like get quite commonly depending on, on its nature is something like pleasure.

Um, again, and then we have to ask how janky is pleasure, how specific and contingent is the thing we care about in pleasure versus how robust is this as a functional role in, in like minds of all kinds.

Um, and I personally don't know, uh, on this stuff.

And I don't think, I don't think this is like enough to get you alignment or something, but I think it's at least worth considering.

uh, being aware of like these, these other features.

We're not sort of talking, we're not really talking about the AI's values in this case.

We're talking about like the kind of structure of its mind and the different properties the minds have.

Um, and I think that, um,

that could show up quite robustly.

So I actually find that it's kind of quite complimentary.

So, yeah.

yeah, I will write these sort of more technical reports and then do this sort of kind of more literary writing and philosophical writing.