Nick Bostrom

It's useful for many goals you might have.

It's useful for many goals you might have.

It requires you to be alive to pursue them.

It requires you to be alive to pursue them.

not strictly for all goals, like there are people who commit suicide, but for most goals that some people have, whether to help the world or to become a despot, like for either of those or for many other goals, take care of your family or enjoy a game of golf, you need to stay alive.

not strictly for all goals, like there are people who commit suicide, but for most goals that some people have, whether to help the world or to become a despot, like for either of those or for many other goals, take care of your family or enjoy a game of golf, you need to stay alive.

So analogously for human, for AIs, that might be sort of instrumental reasons to try to avoid scenarios where they would get shut off.

So analogously for human, for AIs, that might be sort of instrumental reasons to try to avoid scenarios where they would get shut off.

Similarly, they might have instrumental reasons to try to gain more computational resources

Similarly, they might have instrumental reasons to try to gain more computational resources

more abilities so that they can think more clearly.

more abilities so that they can think more clearly.

And in some cases, this might involve instrumental reasons to hide their intentions from the AI developers, particularly if they are misaligned.

And in some cases, this might involve instrumental reasons to hide their intentions from the AI developers, particularly if they are misaligned.

And then obviously revealing those misaligned goals to the AI programmer team might just mean they get reprogrammed or retrained to have those goals erased, and then they won't achieve them.

And then obviously revealing those misaligned goals to the AI programmer team might just mean they get reprogrammed or retrained to have those goals erased, and then they won't achieve them.

And so you could have strategic incentives for deception or for sandbagging or underplaying your capabilities, etc.

And so you could have strategic incentives for deception or for sandbagging or underplaying your capabilities, etc.

So this is a change in regime that makes potentially aligning advanced AI systems

So this is a change in regime that makes potentially aligning advanced AI systems