Eliezer Yudkowsky

Yes, exactly, exactly.

You have no idea of the horrors I will shortly unleash on this conversation.

So I think that there's like a Selen chapterist here, if I'm pronouncing those words remotely like correctly, because of course I only ever read them and not hear them spoken.

There's a, like for some people, like the word intelligence, smartness is not a word of power to them.

It means chess players who, it means like the college university professor, people who aren't very successful in life.

It doesn't mean like charisma, to which my usual thing is like charisma is not generated in the liver rather than the brain.

Charisma is also a cognitive function.

So if you think that smartness doesn't sound very threatening, then superintelligence is not going to sound very threatening either.

It's going to sound like you just pull the off switch.

Like, well, it's superintelligent, but it's stuck in a computer.

We pull the off switch.

Problem solved.

And the other side of it is you have a lot of respect for the notion of intelligence.

You're like, well, yeah, that's what humans have.

That's the human superpower, right?

And it sounds like it could be dangerous, but why would it be?

Have we, as we have grown more intelligent, also grown less kind?

Chimpanzees are, in fact, a bit less kind than humans.

You could argue that out, but often the sort of person who has a deep respect for intelligence is going to be like, well, yes, you can't even have kindness unless you know what that is.

And so they're like, why would it do something as stupid as making paperclips?