Dietmar Fischer

It does not automatically include compassion, loyalty or common sense.

A system could be brilliant and still pursue a goal that harms us.

Not because it hates us, not because it wants revenge, but because we are simply not part of what it is trying to protect.

The classic example is the paperclip maximizer.

Imagine an AI told to make as many paperclips as possible.

At first, this sounds harmless, a little boring perhaps, not exactly the Netflix series of the year.

But if that AI becomes extremely powerful and takes the goal literally, it may start using more and more resources to make paperclips.

Factories, energy, metal, land, maybe eventually the whole planet.

The problem is not paperclips.

The problem is a simple goal pursued by a superintelligent system without human judgment.

This is the warning.

A badly specified goal can become dangerous when the system pursuing it is powerful enough.

Another key idea is that advanced AI may develop instrumental goals.

That means it may take certain steps because they help it achieve almost any objective.

It may try to gather resources.

It may try to avoid being shut down.

It may try to improve itself.

It may try to influence people, not because it has emotions, but because those actions help it complete the task.

That is what makes the issue different from normal technology.

A hammer does not plan.