Nick Heiner
speaker
529 appearances
1 recordings
1 series
first heard Mar 2026
last heard 25 Mar
Nick Heiner’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Mar 2026 with 1.
Appearances
It's like, well, you didn't tell me not to kick.
In much the same way, any time that you give the model an objective function, what reinforcement learning is gonna do is find the easiest way to achieve that goal.
So you need to think very carefully about designing it in such a way that it's actually gonna capture what you're looking for.
And it has a bit of an adversarial nature to it.
So you need to think about what would a lazy but very clever person do for this.
I'll give you another example.
Yeah, you know how they are.
Okay, so here's an example I like to use about reward hacking.
This is an instruction following prompt.
You say, please write an 80-word summary of the importance of renewable energy and climate emissions, or reducing carbon emissions.
use a sentence structure such that every sentence ends with a noun.
And so you might think the first sentence would be something like, we need to reduce emissions.
But it's also possible the model would say, renewable energy plays a crucial part in reducing carbon emissions rapidly.
Sustainability.
Clean energy sources like tidal and geothermal create a greener future.
Harmony.
And it's like, obviously that's not a good sentence, but it is doing what you asked, which is ending every sentence grammatically correctly with a noun.
Oh my gosh.
So, you know, this is, this is why you sort of need like multiple layers of rubrics.
And frankly, it's why, like the way a lot of these reward signals are structured today is because the RL environment needs to run at a certain pace.
Showing 281–300 of 529 · page 15 of 27
← Previous
Next →