Logan Kilpatrick

Made it through some years of school how to play chess and probably 30 minutes and they'd be able to pick it up.

Yeah.

So it is it is fascinating to see that that model behavior play out in that way.

Yeah, this is definitely part of my like joke is, you know, everything comes back to instruction following.

So if you can just make the model better at instruction following, you can do anything.

And like chess is a good example of that.

Like technically, if you were to just make the model good at instruction following, you should be able to give it really, really good instructions.

And it should be able to go in and generalize from that.

The reality is like it's not that easy.

it's not that clear.

And to answer the question specifically, like it's an open research question, like how much does making the models better at these games actually generalize to other capabilities?

Hopefully it does.

And you could imagine as part of the original, like deep mind thesis is you teach the models, a bunch of stuff about all these different environments and the

reinforcement learning environments and you can make them really good at domain specific tasks.

And then ideally you learn a bunch of capabilities that generalize across a bunch of different stuff.

that is turned out to be like somewhat true in certain areas.

And like, maybe we'll end up being more true over time, but it's not like a clear slam dunk.

Like you could make, and there's lots of good examples of this, like actually making an AI system, non LLM frontier model that can play all these games was a problem that was solved probably like four or five years ago.

And yet at that time,

there was nothing like LLMs.