Cal Newport

Here's the best.

Here's the most possible point.

So they didn't replace Erdos' conjecture with a better conjecture.

They didn't prove Erdos' conjecture, but they provided a counterexample construction that showed the thing he thought was the right answer couldn't possibly be right.

Now, how did they do this?

Well, they used a reasoning LLM.

So that's an LLM that has been tuned to essentially talk out loud, to sort of think out loud and wander with its thoughts.

We first saw the first reasoning models back in 2024 with O1 and the O models, a deep sequence reasoning model as well.

Basically, reasoning models are a way of taking an LLM, which are static and have no memory,

and having them approximate something like more dynamic computation with memory, because it can sort of, as it rambles, right?

It's looking at everything it said so far when it produces the new token.

So it can, if you're rambling, you're thinking out loud, you can use all of that thinking in producing the new token.

So it's like you have some memory and this wandering can be somewhat dynamic.

You can get some basic like iterative or looping type thinking in it, right?

So they use the reasoning model.

And what they did is, I don't know how many times they prompted it or on what questions they prompted it, but on one of the times they prompted it about this particular problem, the model spit out a very long transcript of an answer.

And a team of expert mathematicians poured over this answer, and in this long chain of thought transcript, they identified in there...

the core idea that became the counterexample.

So these mathematicians then pulled that counterexample idea out of this transcript, they polished it, they wrote it properly, they elaborated it, and put it into a short, concise, much more human-readable paper, and that's what OpenAI actually posted.

So the LLM did not post this sort of elegant