Richard Sutton

And this is, I think, a major stage in the universe, a major transition, a transition from replicators, humans and animals,

plants we're all replicators and that gives us some strengths and some limitations and then we're entering the age of design where because our ai's are designed our our our all of our physical objects are designed our buildings are designed our our technology is designed and we're we're designing now uh ai's things that can be intelligent themselves and that are themselves capable of design and so this is this is a key step in the world and in the universe and i think

So it's the transition from the world in which most of the interesting things that are, are replicated.

Replicated means you can make copies of them, but you don't really understand them.

Like right now we can make more intelligent beings, more children, but we don't really understand how intelligence works.

Right.

Whereas we're reaching now to having design intelligence, intelligence that we do understand how it works, and therefore we can change it in different ways and at different speeds than otherwise.

And our future, they might not be replicated at all.

We may just design AIs, and those AIs will design other AIs, and everything will be done by design and construction rather than by replication.

Yeah, I mark this as one of the four great stages of the universe.

First there's dust, ends of stars, and then stars make planets, and the planets give rise to life, and now we're giving life to designed entities.

And so I think we should be proud that we are giving rise to this great transition in the universe.

Yeah, so it's an interesting thing.

Should we consider them part of humanity or different from humanity?

It's our choice.

It's our choice whether we say, oh, they are our offspring and we should be proud of them and we should celebrate their achievements.

Or we could say, oh, no, they're not us and we should be horrified.

It's interesting that it feels to me like a choice, and yet it's such a strongly held thing that how can we be a choice?

I like these sort of contradictory implications of thought.

So are you thinking like maybe we are like the Neanderthals who give rise to Homo sapiens.