Nick Bostrom

Just as if you assign a scientist a question that they should try to figure out the answer to, like, how safe is this drug?

Just as if you assign a scientist a question that they should try to figure out the answer to, like, how safe is this drug?

But then in the course of trying to answer that, they might have to make plans and pursue goals, like, oh, how do I...

But then in the course of trying to answer that, they might have to make plans and pursue goals, like, oh, how do I...

you know, get the research grant to fund this research.

you know, get the research grant to fund this research.

How do I hire the right people to work on my research team?

How do I hire the right people to work on my research team?

How do I?

How do I?

And so internally, you could have processes, maybe unintentionally, arising during training within the AI mind itself that could have objectives and long-term goals, even if that was not kind of the function that you wanted the AI system to play.

And so internally, you could have processes, maybe unintentionally, arising during training within the AI mind itself that could have objectives and long-term goals, even if that was not kind of the function that you wanted the AI system to play.

And that can happen with any of these three types.

And that can happen with any of these three types.

If you look at systems that sort of behave as intended, like a simple Oracle system without any safeguards could help answer questions that we don't want people to be able to answer.

If you look at systems that sort of behave as intended, like a simple Oracle system without any safeguards could help answer questions that we don't want people to be able to answer.

Like, how do I make more effective biological weapons?

Like, how do I make more effective biological weapons?

How do I make this hacking tool that allows me to hack into...

How do I make this hacking tool that allows me to hack into...