Yi Tay
speaker
925 appearances
1 recordings
1 series
first heard Jan 2026
last heard 23 Jan
Yi Tay’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.
Appearances
Is attention all you need?
So attention and transformers have been is a like a core idea in the recent times.
Like pre-training and scale is the thing that made attention and transformers like actually shine, right?
Because it without I think the first transformer paper was this like a machine translation thing.
And then basically GPT and BERT were the ones that like actually showed the full
like big potential of this idea.
So in terms of is attention like really, really all we need?
Like probably no.
But I think it's like one of the from the architectural point of view, also may maybe no.
But it's not all you need, it's but you we need it definitely.
I think
You'll be a transform I think.
Really.
Like like it we like people c it depends on what you call it.
But I think unless the paradigm shifts completely, which is I mean as a scientist you cannot like like completely like say no to like like that d this will never happen.
But my feeling is that it's been like what like almost ten years since transform.
Twenty seventeen.
I think we have not r replaced self-rettenation.
Like it's uh in some form of it.
Like you could rename it, you could name it something else, you could something.
Showing 481–500 of 925 · page 25 of 47
← Previous
Next →