Cal Newport
👤 SpeakerAppearances Over Time
Podcast Appearances
And the thing GPT-4 did that really got... So GPT-4 set off the whole industry.
The thing it did is it started showing abilities beyond just language.
And that's where people got excited.
Like, oh, wow.
If you train a language model on enough language, it learns about things that isn't just producing language.
It can play games.
It can do math problems.
It can do logic.
I mean, this was super exciting.
It was super exciting.
So the assumption was...
Do this two or three more times, you have AGI.
So that's what the whole industry was based off of.
When we went from three to four, this was legitimate, justified excitement.
Expand the size and the training duration two or three more times, and the economy is going to happen in a box.
I mean, that was the engine for all this excitement.
So they tried.
At OpenAI, it was called Project Orion.
They made it bigger, modeled in four.
They trained it even longer.