Illia Polosukhin

And I mean, there's actually like, you know, improvements, like there's a new optimizer came out that, you know, like effectively for many years, there was this atom that was used.

And so now there's like this Muon new optimizer, which like, you know, has different algebraic properties around how gradients are propagated.

And, you know, there's like probably, I don't know, 50 people in the world that actually understands exact like mechanism of that beyond like it's magic.

But, you know, it literally shows that model just trains faster.

Like same data, same model size trains faster, right?

So clearly we haven't squeezed out everything we can from just how we're training these models.

So that's kind of like, I would say like big area for me and I'm excited in that.

I think the other one is just context.

A lot of these models are actually really good now, but they just don't know stuff.

They don't know stuff about you.

They don't know stuff about your company and your work you're doing.

They don't know things about the world that's happening right now.

And yeah, there's some search, some other tools, but there's a very limited context that the model has.

Yeah.

And I think this is really probably the bigger bottleneck for actual adoption in personal and even more importantly, kind of work environments.

And I think one of the areas where, I mean, to bring it back to what we're working on, where I think is limitation, it's...

like there's kind of both sides are a little bit scared of of allowing this to happen so like i'm as a user scared of like allowing access to all of my data 100 and then the company is actually on the other side also a little bit scared of like getting all of the user data and be responsible for you know something goes wrong and so that's true and who knows what it is yeah

Yeah, like, again, you know, there's a HIPAA compliance, right?

If you're taking on medical data, like, for example, all of the AI companies now, people uploading their medical data there, and they're not HIPAA compliant.

So this is like a big question there, right?