Nathan Lambert
speaker
1,013 appearances
1 recordings
1 series
first heard Nov 2024
last heard Nov 2024
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
And you have to do this.
And then those people that goes to a training person.
Training person does DPO, they pass it to RL.
So you kinda just have these people that naturally
Um
To zoom out, you have people that naturally converge to different areas.
And this is probably like ten to fifteen people that are actively involved most of the time.
At this size, we did not need strict delegation in management.
If you go to the llama size, a hundred percent chance you need delegation in management and rules who who makes what decision when.
Effectively I'm like softly that person, which is like making the call on what SMT mix is final.
But there's definitely an interesting transition organizationally, which is like you cannot scale it more because it is uh becomes a mess if you're more than 10 to 15 contributors on one of these language modeling processes.
But then there becomes a big cost if you add in managers, because then you have to do a lot more of like delegation of decision making and stuff like this.
So in some ways you can tell by how I was describing it, a lot of this is like somewhat chaotic and free form and just relying
on people being
in the weeds, in the details, and very happy to communicate with the various people that they know needing it.
And a lot of that is autonomous, it's just like
Um I
can't be over Valentina's shoulder being like, get your DPO model to Haymish to do RL on.
It's like they just do that.
And
Showing 641–660 of 1,013 · page 33 of 51
← Previous
Next →