Dwarkesh Patel
speaker
19,288 appearances
62 recordings
3 series
first heard Feb 2024
last heard 5d ago
Dwarkesh Patel’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 27 in all, peaking in Jun 2026 with 6.
Appearances
Okay, the third common objection people make is that we just haven't scaled enough.
We have these scaling laws.
They tell us the bigger models are more sample efficient.
The human brain we know is about 100 trillion synapses.
And we have frontier models that are currently around 5 trillion parameters.
And so maybe we could just achieve human-level sample efficiency if we made these models one to two orders of magnitude bigger.
The reason this objection is off-mark is actually quite interesting.
So if you look at the way the scaling loss equations work, they tell you that the parameter and data terms are added to the loss independently.
So suppose you have a model and you've trained it to compute optimally and you say, I want to be sample efficient.
I want to use as little data as possible and I'll throw in as many parameters as is necessary to make that happen.
So take the constants from the Chinchilla Scaling Law paper.
Even if you increase the number of parameters by infinity, that would only decrease by a factor of 10 the amount of data that you need in order to keep the same loss.
Humans are somewhere between thousands to millions of times more sample efficient than these models.
So scaling the size of current models simply can't make up for that discrepancy.
And this really does suggest that humans are in a different scaling curve altogether.
As soon as I earn money, I want to put it to work.
But I also need to save for things like upcoming expenses and estimated taxes.
So to figure out exactly how much I need to set aside, I ask Command.
Command is AI that is built into Mercury, which is my banking platform.
And since I already use Mercury to run my entire business, Command has access to all the information it needs to get work done.
Showing 3561–3580 of 19,288 · page 179 of 965
← Previous
Next →