Illia Polosukhin
speaker
1,856 appearances
82 recordings
16 series
first heard Mar 2020
last heard 5d ago
Illia Polosukhin’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 47 in all, peaking in Jul 2026 with 8.
Appearances
There's nothing to train on, like at scale.
But now we do have this, right?
Now we actually have a lot of this agentic interactions now that everybody's running.
So there's actually data now to train this like longer range task.
I'm pretty confident, yeah.
I mean, you know, as I talk with researchers, this is probably one of the main challenges that everybody's targeting right now.
Yeah, I mean, effectively, the way to think about it, I mean, we can go physiological where, you know, the human learn, whatever.
In the span of years, yes, you only maybe...
have like 80 million tokens in 10 in a decade right so it's not like you're actually not getting that much like language tokens but you have visual tokens you have like you have physical you have all of this additional information and that actually is like our what what kind of goes from a pre-trained model we are born with right to a
fully, fully fine-tuned, you know, people we are.
And so AI right now, yeah, as I said, like it's a, it's just like genius in the memento, in the memento state, right?
And so to really unshackle it more, you kind of really need to, this,
longer context and like it already has ability to learn in context so this like concept of in context learning right so if you if you show it something it didn't know about before it will start using it but it needs to be in the context and so you know as you show it like here's the thing i want you to do and then you know like it goes does a bunch of stuff all of that is fills its context and now
like, again, all the actions, all the responses, like, if it read an article about, you know, for example, preparing for this interview, it went, read an article from Nir, like, all of that now is in its context, right?
And, like, there's techniques to kind of compress it, summarize it, you know, have sub-agents to do a bunch of stuff.
So there's, like, different ways to, like, mitigate it, but at the end, still, like, at some point, it's like, okay, I'm out of context, and now to do next thinking...
I need to clear stuff up.
I need to remove something.
Yeah.
And at that moment, it's a very lossy because it doesn't actually know what's useful, what's going to be useful in going forward.
Showing 641–660 of 1,856 · page 33 of 93
← Previous
Next →