Dwarkesh Patel

First, pair the layers into 48 different blocks.

And second, put those blocks in the right order.

For pairing, Sean realized that in a well-trained resonant, the product of two weight matrices in a residual block should have a distinctive negative diagonal pattern.

And this arises as a way for the model to keep the residual stream from growing out of control.

From this insight, he was able to recover the right pairings.

For ordering, Sean noticed that the model seemed to improve if he sorted the blocks by the size of their residual contributions.

Starting with that rough approximation, he combined a clever ranking heuristic with local swaps to recover the exact right order.

His full walkthrough is linked in the description.

Don't worry if you didn't get to this puzzle in time, though.

There's still one up about backdoor LLMs that even Jane Street doesn't know how to solve.

You can find it at janestreet.com slash thwarkash.

All right, back to Terrence.

Okay, so I think this brings us nicely to

the progress that from the outside, it seems like AI for math is making.

And I think you had a post recently where you pointed out that over the last few months, AI programs have solved 50 out of the 1100 odd for those problems.

But then I think, I don't know if it's still correct, but as of a month ago, you said that there had been a pause because the low hanging fruit had been picked.

First of all, I'm curious if actually that is still the case that

we have picked a low-hanging fruit and now we're at this plateau currently.

So there's two different ways to think through what you've just said.

And one of them is more bearish on AI progress and one of them is more bullish and bearish on being, oh, they're only getting to a certain height of wall, which is not as high as humans are reaching.