Cal Newport

And so making progress on one tributary doesn't really tell you anything about whether this other tributary down the river is going to be equally as explorable or not.

That is what I've been arguing is the right model.

And I think that helps us better place these math results in context.

And remember, ever since like ChatGPT put generative AI and large language models on people's radar, computer scientists have been saying there are two areas in particular in which LLMs are going to be very well suited.

computer programming and mathematical reasoning.

And this is because those two problems share four elements.

They deal with highly structured language, either computer code or mathematical notation.

They have clear notions of correctness.

Does this program compile and pass the test?

Is this proof true?

Is this math result right?

There's endless data to train on.

The computer programming tuned models, we can tune it on these sort of endless examples of code online where people ask questions, other people give answers.

Math is even better because you can actually artificially, synthetically create data, example after example of many different math problems and correct proofs and tune it again and again and again with it.

So it's really good at particular types of mathematical reasoning.

And also in both cases, programming and math, you have expert users who are willing to use hard tools and massage good results out of it.

If you told me I have this great AI tool, it's going to spit out 150 pages.

You have to comb through it and try to piece together.

Maybe in there, there's a useful insight for like my business problem.

The average business person is going to say, I'm still trying to figure out where the paperclip and Microsoft Word went.