Yi Tay

speaker

925 appearances 1 recordings 1 series first heard Jan 2026 last heard 23 Jan

Yi Tay’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.

newest first · ▶ plays the moment

Yeah, it's still a transformer in the end.

And I think like that's not going like anywhere.

Unless the whole thing with like back prop, like everything like goes like the whole thing just changes completely.

Like then there's a different story, that's a different conversation to have.

But if it's still within the

same scope and bounds.

So I spent a lot of time thinking of about architectures and like whether there's alternate architectures and stuff like that.

Okay, at the sequence processing level.

Like there's There is the ultimate Yeah, it's sequence to sequence transformer.

It's probably the self-attention is there was this whole big era, which I was also involved in this era where people tried to like

undermine the attention as much as possible.

Like they try to remove it, simplify it, make it efficient, like this whole like efficient attention era.

At the end of the day, the outcome was always like, oh we remove all the attention, but we have one layer of self-attention there and it still works.

Like that's at the end like of the always the story.

consider local and global attention to be lights still still attention.

Just like how

you're skipping.

Yeah, yeah.

Uh but it's that's not really about sequence and sequence, that's more about like the whole

b gradient design and backprop thing, right?

Showing 501–520 of 925 · page 26 of 47