Konstantin Kisin

We don't align it to the end user.

Sometimes it says the end user, you can't make bioweapons.

We don't align it to the model company.

We shouldn't have model companies that control superintelligence.

The government doesn't get to just say whatever this future army should do.

Then what's the fourth option left?

The AI itself.

So should AI systems have their own values, their own sense of morality?

And look, I know this... I remember reading some Isaac Asimov books that went very much like this.

And of course, it sounds like a terrible, you know, this sounds like I'm describing the Terminator, right?

Like the Terminator has its own values.

No, it obviously matters a ton what those values should be.

And I think that in the future, there will need to be the equivalent of a constitutional convention where we get together and the best political philosophy, the best thought that has happened in human civilization needs to go into thinking about what are the checks and balances?

What are the exact values?

What are the situations in which an AI should do X versus Y?

When should it refuse to do something?

What are the laws of robotics?

Exactly.

As Asimov talked about.

Exactly.