Yoshua Bengio
speaker
2,216 appearances
5 recordings
5 series
first heard Oct 2018
last heard 7 May
Yoshua Bengio’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 3 in all, peaking in May 2026 with 1.
Appearances
But even if you don't have a mathematical guarantee, it might still be a workable approach.
So I think it's worth doing.
In other words, we can take a really competent top-notch model
and then continue training using the objective of the scientist AI and the data that has been transformed as I discussed.
And we hope to show empirically that.
As you do more and more fine tuning,
the measurements of honest behavior, lack of deceptive behavior will improve and that we won't lose in capability.
So that wouldn't be like a mathematical proof.
That would be like an empirical thing.
And once this is established, then it might be sufficient to convince people, okay, let's get the full guarantees by training from scratch, which is now gonna cost the cost of training full model.
So first, let me talk about reinforcement learning.
Three years ago, I was in a meeting with a bunch of reinforcement learning researchers, and I had a slide with only these words, reinforcement learning is evil.
But what do you really think?
So, you know, this is not something new.
People in AI safety have been talking about the fundamental flaw in training by reinforcement learning to achieve something in the world.
It gives rise to the problems of instrumental goals and reward hacking.
And in both of these cases, what you do end up with is systems that have goals that you didn't choose and could go against the goals that you did choose.
So reinforcement learning is a very dangerous thing to build superintelligence.
The good news is you don't need to do reinforcement learning.
Like what we show with the scientist AI is that there's a way to train the AI so that it will be indifferent to the consequences of its actions.
Showing 401–420 of 2,216 · page 21 of 111
← Previous
Next →