Marcus Hutter

So the theory is really great.

With the ICSI model, with the planning part,

Many results are only asymptotic, which, well, this is... What does asymptotic mean?

Asymptotic means you can prove, for instance, that in the long run, if the agent acts long enough, then it performs optimal or some nice thing happens.

But you don't know how fast it converges.

So it may converge fast, but we're just not able to prove it because it's a difficult problem.

Or maybe there's a bug in the model so that it's really that slow.

So that is what asymptotic means, sort of eventually, but we don't know how fast.

And if I give the agent a fixed horizon M, then I cannot prove asymptotic results, right?

So I mean, sort of if it dies in 100 years,

then 100 years is over, I cannot say eventually.

So this is the advantage of the discounting that I can prove asymptotic results.

It's like with the playing chess, right?

You do this minimax.

In this case here, do expectimax based on the Solomonov distribution.

You propagate back, and then while an action falls out, the action which maximizes the future expected reward under Solomonov distribution, and then you just take this action.

And then repeat.

And then you get a new observation and you feed it in this action and observation, then you repeat.

So when I started in the field, I was always interested in two things.

One was AGI.