Sam Harris

You know, fallaciously or not, I'll grant you that all of this is made up and it's hard to come up with a rational estimate, but for the people doing the work, plowing, you know, trillions of dollars into the build-out of AI to be giving numbers like 20% seems culturally strange.

Do you have any thoughts about how a system would have to be built so as to be perpetually aligned with our interests?

I mean, if you're taking intelligence seriously, right?

So we're talking about building an autonomous intelligence system that exceeds our own intelligence and in the limit improves itself, one would imagine.

Do you have any

notions about what a guarantee of an alignment could look like before we hit the play on that?

I think Stuart Russell, it's been a couple of years since I've spoken with him, but I recall his notion, again, I'm sure this is kind of a hand-waving notion from the computer science point of view, but to have as its utility function just to better and better approximate what we want.

to be perpetually uncertain that it's achieved our goals insofar as we can continue to articulate them in this open-ended conversation that is the evolution of human culture.

So people have an intuition that if we built it, there's no possibility of it forming its own goals that we didn't anticipate, the instrumental goals.

I mean, there are people fairly close to the field who will say this.

I'm not sure.

I mean, maybe even someone like Jan LeCun would say this.

But what would you say to that?

I mean, you just very breezily articulated certainty that or something like certainty that an independently intelligent system can play, that it can change its mind, it can discover new goals and cognitive horizons, just as we seem to be able to do.

Why is there a difference of intuition on this front?

I mean, your account seems obvious to me.

All right, so this is kind of a reckless pivot from the topic of AI, but I think there's a bridge here.

I mean, I guess we could put this sort of in the frame of the cultural conditions that allow us to reason effectively or fail to reason effectively.

And this is on morally loaded topics like war and asymmetric violence, anti-Semitism, Islamism, again, Israel status among nations.

Unfortunately, you are unusually well-placed to have an opinion on these topics, given your history and what happened to your son back in 2002.