Tristan Harris

In the case of the military weapons, we end up in crazy Terminator-like war scenarios that no one actually wants.

In the case of businesses, we end up disrupting billions of jobs and creating mass outrage and public riots on the streets because people don't have food on the table.

And so much like climate change or these kind of collective action problems or the ozone hole, we're kind of creating a badness hole through the results of all these individual competitive actions that are supercharged by AI.

But if that war escalates into... I mean, the reason that the Soviet Union and the United States have never directly fought each other is because the belief is it would escalate into World War III and nuclear escalation.

If China and the US were ever to be in direct conflict, there's a concern that you would escalate into nuclear escalation.

So it looks good in the short term, but then what happens when it...

sort of everything gets chain-reactioned into everybody escalating in ways that causes many more humans to die.

But how is it if the automated weapon gets fired...

And it leads to, again, a cascade of all these other automated responses.

And then those automated responses get these other automated responses and these other automated responses.

And then suddenly the automated war planners start moving the troops around.

And suddenly you've created this sort of escalatory loss of control spiral.

Yeah.

And then humans will be involved in that.

And then if that escalates, you get nuclear weapons pointed at each other.

It's a narrow boundary analysis on whereas this machine, you could have put a human at risk.

Now there's no human at risk because there's no human who's firing the weapon.

It's a machine firing the weapon.

That's a narrow boundary analysis without looking at the holistic effects on how it would actually happen.

Which we're bad at.