Alex Wissner-Gross

Transformers turn out to be amazing generalist models at solving general tasks.

So we saw, as I mentioned earlier, language models are few shot learners.

The GPT-3 paper show us that transformers can solve generally intelligent tasks.

So then we see the boom of transformers in general.

Fast forward another few years, circa, and again, the timelines get a little bit fuzzy because reasoning models were in some sense in the air, but hadn't quite congealed, but call it say 2024 or so, when we start to see the emergence of the next generation reasoning models, starting

principally with OpenAI's O1 model.

Some would quibble and say there were a few models that demonstrated this principle beforehand, but the idea with reasoning models is not just asking a large language model to complete the next word, complete the next token, but actually giving it a bit of space and time to think before it produces an answer, and then training it specifically to leverage the space and time to think, to have an internal monologue, if you will.

What does that mean?

You just asked me a question.

What does that mean?

Before I answer, even though I started my answer pretty quickly, I have an internal monologue.

I'm thinking about what is the best way to respond to that question.

I'm burning my internal GPUs, if you will, trying to formulate the best possible answer to that question before I give you my final response.

That's what a reasoning model is.

Rather than instantaneously responding, so-called system one thinking, thank you Kahneman, it uses system two thinking to have an internal discussion with itself and think about what it should say before it says it.

And that's what has unlocked math and physics, arguably, could talk more about that.

The sciences in general.

And now we're at an era where reasoning models, and we've seen now everyone adopt reasoning models, no longer just large language models, but multimodal reasoning models, are poised to solve the hardest problems on Earth.

You're asking, in some sense, the person who has inadvertently become probably a leading public evangelist for some form of AI personhood.

These are questions I ask all the time.