Ilya Shumailov

I've got to be absorbing this bias.

Exactly. And those are the kinds of errors that you don't normally see that often because they are so improbable, right? And if people are going to start reporting things to you and saying, oh, your model is wrong here, they're likely to notice things that on average are wrong.

Exactly. And those are the kinds of errors that you don't normally see that often because they are so improbable, right? And if people are going to start reporting things to you and saying, oh, your model is wrong here, they're likely to notice things that on average are wrong.

Exactly. And those are the kinds of errors that you don't normally see that often because they are so improbable, right? And if people are going to start reporting things to you and saying, oh, your model is wrong here, they're likely to notice things that on average are wrong.

But if they're wrong in some small part of the internet that nobody really cares about, then it's very unlikely that you will even notice that you're making a mistake. And usually this is the problem because... As the number of dimensions grow, you will discover that the volume in the tails is going to grow disproportionately.

But if they're wrong in some small part of the internet that nobody really cares about, then it's very unlikely that you will even notice that you're making a mistake. And usually this is the problem because... As the number of dimensions grow, you will discover that the volume in the tails is going to grow disproportionately.

But if they're wrong in some small part of the internet that nobody really cares about, then it's very unlikely that you will even notice that you're making a mistake. And usually this is the problem because... As the number of dimensions grow, you will discover that the volume in the tails is going to grow disproportionately.

Yeah, exactly. So as a result, you'll discover that you need to capture quite a bit.

Yeah, exactly. So as a result, you'll discover that you need to capture quite a bit.

Yeah, exactly. So as a result, you'll discover that you need to capture quite a bit.

On top of it, we have errors that come from learning regimes and from the models themselves. So on learning regimes, we are all training our models. All of them are structurally biased. So basically to say that your model is going to be good, But it's unlikely to be optimal. So it's likely to have some errors somewhere. And this was the error source number two.

On top of it, we have errors that come from learning regimes and from the models themselves. So on learning regimes, we are all training our models. All of them are structurally biased. So basically to say that your model is going to be good, But it's unlikely to be optimal. So it's likely to have some errors somewhere. And this was the error source number two.

On top of it, we have errors that come from learning regimes and from the models themselves. So on learning regimes, we are all training our models. All of them are structurally biased. So basically to say that your model is going to be good, But it's unlikely to be optimal. So it's likely to have some errors somewhere. And this was the error source number two.

And error source number three is that the actual model design, what shape and form your model should be taking, is very much alchemy. Nobody really knows why stuff works. We kind of just know empirically stuff works.

And error source number three is that the actual model design, what shape and form your model should be taking, is very much alchemy. Nobody really knows why stuff works. We kind of just know empirically stuff works.

And error source number three is that the actual model design, what shape and form your model should be taking, is very much alchemy. Nobody really knows why stuff works. We kind of just know empirically stuff works.

Yeah, which parts of the model are responsible for what? We don't know the fundamental underlying bias of a given model architecture. What we observe is that there is always some sort of an error that is introduced by those architectures.

Yeah, which parts of the model are responsible for what? We don't know the fundamental underlying bias of a given model architecture. What we observe is that there is always some sort of an error that is introduced by those architectures.

Yeah, which parts of the model are responsible for what? We don't know the fundamental underlying bias of a given model architecture. What we observe is that there is always some sort of an error that is introduced by those architectures.

Exactly. And then we also have empirical errors from, for example, hardware. So we also have practical limitations of hardware with which we work. And those errors also exist.