Nick Bostrom

You can do that your whole life.

You can do that your whole life.

It involves many specific little tasks, like trying to raise money for this charity and trying to launch this new campaign or trying to invent some new medicine that will help.

It involves many specific little tasks, like trying to raise money for this charity and trying to launch this new campaign or trying to invent some new medicine that will help.

All of these would be sort of subtasks, but it's in pursuit of this open-ended objective.

All of these would be sort of subtasks, but it's in pursuit of this open-ended objective.

So similarly, you could have an AI system, maybe internally, it's like a unified, simple agent architecture, but that is operating in pursuit of such open-ended objective.

So similarly, you could have an AI system, maybe internally, it's like a unified, simple agent architecture, but that is operating in pursuit of such open-ended objective.

Conversely, like even an Oracle that just tries to answer a question internally, theoretically, it could be a multi-agent architecture.

Conversely, like even an Oracle that just tries to answer a question internally, theoretically, it could be a multi-agent architecture.

We have different sort of research agents that get sent off to answer different sub-questions in order then to combine at the end to produce an answer to the user.

We have different sort of research agents that get sent off to answer different sub-questions in order then to combine at the end to produce an answer to the user.

One has to distinguish the internal architecture of the system from the role that it is designed to play in society.

One has to distinguish the internal architecture of the system from the role that it is designed to play in society.

Yeah, so... They all share a bunch of things that could go wrong with all of them, which is...

Yeah, so... They all share a bunch of things that could go wrong with all of them, which is...

However they are intended to operate, they might not actually operate that way.

However they are intended to operate, they might not actually operate that way.

So you might construct an AI that you intend to serve just as a question answering system, but then internally it might have goal seeking processes

So you might construct an AI that you intend to serve just as a question answering system, but then internally it might have goal seeking processes