Dwarkesh
speaker
1,722 appearances
5 recordings
1 series
first heard Apr 2025
last heard 25 Nov
Dwarkesh’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in Nov 2025 with 1.
Appearances
So we think that this latter category of training failure is going to get much worse as they become agents.
Agency training, you're going to reward them when they complete tasks quickly and successfully.
This rewards success.
There are lots of ways that cheating and doing bad things can improve your success.
Humans have discovered many of them.
That's why not all humans are perfectly ethical.
And then you're going to be doing this alternative training where afterwards for one-tenth or one-one-hundredth of the time, like, yeah, don't lie, don't cheat.
So you're training them on two different things.
First, you're rewarding them for this deceptive behavior.
Second of all, you're punishing them.
And we don't have a...
great prediction for exactly how this is going to end.
One way it could end is you have an AI that is kind of the equivalent of the startup founder who really wants their company to succeed, really likes making money, really likes the thrill of successful tasks.
They're also being regulated and they're like, yeah, I guess I'll follow the regulation.
I don't want to go to jail.
But it's not like robustly, deeply aligned to, yes, I love regulations.
My deepest drive is to follow all of the regulations in my industry.
So we think that an AI like that, as time goes on and as this recursive self-improvement process goes on, will kind of get worse rather than better.
It will move from kind of this vague superposition of, well, I want to succeed.
I also want to follow things to like being smart enough to genuinely understand its goal system and being like, my goal is success.
Showing 1341–1360 of 1,722 · page 68 of 87
← Previous
Next →