Dietmar Fischer

A badly specified goal can become dangerous when the system pursuing it is powerful enough.

Another key idea is that advanced AI may develop instrumental goals.

That means it may take certain steps because they help it achieve almost any objective.

It may try to gather resources.

It may try to avoid being shut down.

It may try to improve itself.

It may try to influence people, not because it has emotions, but because those actions help it complete the task.

That is what makes the issue different from normal technology.

A hammer does not plan.

A car does not persuade you not to repair its brakes.

A database does not build a strategy to stop you unplugging it.

But an advanced AI agent could plan, adapt, use tools, and act in the world.

For Yudkowsky, that changes everything.

A chatbot that answers questions is one thing.

An AI agent that can write code, send messages, use software, manage tasks, call APIs and make decisions is another.

The more autonomy and access we give these systems, the more important alignment becomes.

And this is why Yudkowsky is not reassured when people say, but today's AI still makes mistakes.

His answer would be, current weakness does not prove future safety.

A tiger cub is also not very dangerous.

That does not mean you should raise one in the office kitchen next to the oat milk.