Nathan Lambert
speaker
1,814 appearances
3 recordings
2 series
first heard Feb 2025
last heard 1 Feb
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in Feb 2026 with 1.
Appearances
For something like a singularity to happen, I would say yes.
And then new ideas can be architectures or training algorithms, which is like fundamental deep learning things.
But they're in that nature pretty hard to predict.
But I think we will get very far even without those advances.
Like we might get this software solution, but it might stop at software and not do computer use without more innovation.
So I think that it's like...
A lot of progress will be coming, but if you're going to zoom out, there's still ideas in the next 30 years that are going to look like that was a major scientific innovation that enabled the next chapter of this.
And I don't know if it comes in one year or in 15 years.
If scaling laws are fundamental in deep learning, I think the bitter lesson will always apply, which is compute will become more abundant.
But even within abundant compute, the ones that have a steeper scaling law slope or a better offset, like this is a 2D plot of performance and compute.
And even if there's more compute available, the ones that get 100x out of it will win.
The problem with that is heat dissipation.
So you get all the radiation from the sun and you don't have any air to dissipate heat.
But there is a lot of space to put clusters.
There's a lot of solar energy there and you could figure out the heat dissipation.
But there is a lot of energy and there probably could be engineering will to solve the heat problem.
So there could be.
When you look at the models, there's so much obvious things to improve and it takes a long time to train these models and to do this art and that it'll take us with the ideas that we have multiple years to actually saturate in terms of whatever benchmark or performance we are searching for.
It might serve very narrow niches, like the average ChatGPT 800 million user might not get a lot of benefit out of this, but it is going to serve
different populations by getting better at different things.
Showing 941–960 of 1,814 · page 48 of 91
← Previous
Next →