Nathan Lambert
speaker
1,013 appearances
1 recordings
1 series
first heard Nov 2024
last heard Nov 2024
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
Um
both SFT, DPO, and RL.
The interesting thing with RL is that we don't quite know how the ceiling is defined per base model.
So if you do RL on like a less good SFT model, we have found that as like on Llama, for example, an 8B, like RL for GSM AK will always saturate at eighty seven to eighty eight GSMAK.
That's just like a fundamental limit of the base model.
No matter what we do at SFT or DPO, we could then get the GSMAK to something like 85 reliably without too much degradation.
On different Olmo-based models, like we take the Ulmo model for July, it's like we bumped it from 60 to 75.
And it's like we don't know what defines these kind of saturation limits, but as you get better at having more training stages, a lot of these kind of like just take it off the shelf and train.
will look different.
So it's like if we know we can really boost GSM AK, it's like we don't need that SFT training data.
And maybe we can use that SFT budget in a different way.
But that type of sophistication is something that we haven't explored a lot of.
So that's why it's like it's kind of a in my mind it's like the to reflect on this and try to think about what you do next.
It's like there's a very different thought process if you like really know you can recover certain abilities at different times.
Where it's not just like argmax at every stage.
And that takes a lot more experimentation that we haven't done.
Um, it depends on the interface you use.
If you really can always just choose the right model, you can do one one per.
I think there's a lot of interest in general interfaces right now.
Um, some of this will change in the RL stage.
Showing 901–920 of 1,013 · page 46 of 51
← Previous
Next →