Nick Bostrom

very many simulations over the course of these sort of mature civilizations might last for billions of years, right?

very many simulations over the course of these sort of mature civilizations might last for billions of years, right?

And you might think that there would be diminishing returns to running scientific simulations.

And you might think that there would be diminishing returns to running scientific simulations.

Like the first simulation you learn a lot, the next thousand you learn a bit more.

Like the first simulation you learn a lot, the next thousand you learn a bit more.

But after you've already run billions of simulations, maybe the incremental gain from running a few more starts to plateau.

But after you've already run billions of simulations, maybe the incremental gain from running a few more starts to plateau.

Whereas there might be other reasons for running simulations that wouldn't be subject to the same diminishing returns.

Whereas there might be other reasons for running simulations that wouldn't be subject to the same diminishing returns.

If that's the case, you might think most simulations they run would be ones driven by other motives than the scientific one.

If that's the case, you might think most simulations they run would be ones driven by other motives than the scientific one.

Yeah, like if they play some intrinsic value on simulations, for instance, that would be one example of a motive that might not saturate in the same way.

Yeah, like if they play some intrinsic value on simulations, for instance, that would be one example of a motive that might not saturate in the same way.

Yeah, it's not so much levels, but more types.

Yeah, it's not so much levels, but more types.

Okay.

Okay.

So an oracle AI basically is a question answering system.

So an oracle AI basically is a question answering system.