Joe Carlsmith

Here's a pattern of reasoning that I think you want to watch out for, is to say, in my role as creator, or sorry, in my role as creation, say you're thinking of humans in the role of creation relative to an entity like evolution or monkeys or mice or whoever you could imagine inventing humans or something like that, right?

You say, I'm qua creation, I'm happy, right?

that I was created and happy with the misalignment.

Therefore, if I end up in the role of creator,

and we have a structurally analogous relation in which there's misalignment with some creation, I should expect to be happy with that as well.

I think I have a much better grip on what's going on with Lewis than with Nietzsche there, so maybe let's just talk about Lewis for a second.

And we should distinguish two... There's a kind of version of the singularity that's specifically a hypothesis about feedback loops with AI capabilities.

I don't think that's present in Lewis.

I think what Lewis is anticipating, and I do think this is a relatively...

simple forecast is something like the culmination of the project of scientific modernity.

So Lewis is kind of looking out at the world and he's seeing this process of kind of increased understanding of a kind of the natural environment and a kind of corresponding increase in our ability to kind of control and direct that environment.

And then he's also pairing that with

a kind of metaphysical hypothesis or, well, his stance on this metaphysical hypothesis, I think is like kind of problematically unclear in the, in the book, but there is this metaphysical hypothesis, um, naturalism, which says that, uh, humans too, and kind of minds, beings, agents are a part of nature.

And so, uh, insofar as this process of scientific modernity involves a kind of

uh, progressively greater understanding of an ability to control nature, um, that will presumably at some point grow to encompass, uh, our own natures and, uh, our, and, and kind of the natures of other beings that in principle we could, um, uh, we could create.

Uh, and Lewis views this as a kind of cataclysmic event in crisis.

Um, you know, part of what I'm trying to say in, and that in particular that it will lead to all these kind of tyrannical, uh, kind of,

and kind of tyrannical attitudes towards morality and stuff like that.

And part of what I'm trying to... Unless you believe in non-naturalism or in some form of kind of Tao, which is this kind of objective morality.

So we can talk about that.