Yi Tay
speaker
925 appearances
1 recordings
1 series
first heard Jan 2026
last heard 23 Jan
Yi Tay’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.
Appearances
Mm-hmm.
It's not the architecture itself.
That's the that's a problem that we that it's more of like the learning paradigm itself rather than the architecture itself.
I think the architecture is just basically like the interface between the learning algorithm and the tokens.
I think it's more about the learning algorithm itself.
And this conto like continued learning this like
There's many ways to think about processing many like insanely large like context, right?
Like two hundred million, one billion tokens or something like that, right?
Like whether it's gonna be like you have a new l learning algorithm that every time you run inference you
you learn on it, right?
Then you can actually have some kind of memory.
Like human beings learning as I'm talking to you, right?
So that that's also like one way.
The other way is like whether okay, maybe somebody will say that, okay, the attention is like this too expensive for two hundred million
one billion context so we need new architecture or some people will say that oh we just improved the
Chips, like accelerators.
So I think many ways to interpret it, but I think it's like i if it's about that there's a lot of fundamental things that if it's about continual learning and stuff that there's a lot of fundamental things about the learning algorithm and and stuff as well, yeah.
That will have to change.
I think like the learning paradigm and architecture and stuff like that goes like hand in hand.
And I think as the field progresses and ideas just stack on top of one another, right?
Showing 521–540 of 925 · page 27 of 47
← Previous
Next →