Nathan Lambert
speaker
1,013 appearances
1 recordings
1 series
first heard Nov 2024
last heard Nov 2024
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
Um code is something we did a bit, but it wasn't the biggest focus.
I think our vowels for code were mostly saturated pretty early on.
But kind of shifting this post-training into a much bigger domain of
economic progress and kind of
Honestly, there's been a slow in how often people are releasing models and data sets and kind of like reboot that.
I think the DPO era was a very fun one for seeing open post training models.
It was c it was bonkers.
Like every week there was a new like DPO model that was state of the art on something.
But that was like last December, last January, and it feels much slower now.
It's much more big players dominating the AI narrative and it's trying to reset that.
They I would say they share an outline of their method.
They don't have a lot of th like I might have hyperparameters, but not really clear without noting their code base and things like this.
So they share a high level outline of like here is our approach and here are the types of things that we do, but not the specific things that we do, and definitely not the data.
So the biggest differentiator for our post training is that we release the data.
I think we built like three to six new data sets just for this project and we're using those with a combination of already existing data sets and we'll release all of them at once.
But really what you're saying is right, is like the development model here was we had a set of evaluations we cared about.
that we tried to cover things like factuality, knowledge, math, code, instruction following, safety.
And the goal the primary development target was Llama 3 A B, where we effectively trained about a thousand A B models to try to get these scores to be high higher at 8B and then validate them at 70 B.
And we're also adding like OLMO models, which is our
like makes sense.
Showing 61–80 of 1,013 · page 4 of 51
← Previous
Next →