Nathan Lambert

speaker

1,013 appearances 1 recordings 1 series first heard Nov 2024 last heard Nov 2024

Nathan Lambert’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

No recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.

Appearances

newest first · ▶ plays the moment

"The Cognitive Revolution" · Everything You Wanted to Know About LLM Post-Training, with Nathan Lambert of Allen Institute for AI · 21 Nov 2024 podcast

It's like OpenAI and Google and stuff release these new API models every couple months.

Whereas Llama is like you get like one or two llamas a year and they drop the weights and it's final.

So it's like kind of interesting to think about the distributions and there's

I mean there's a lot of policy discussions on like weights being final and out there, but like that type of stuff is also informing the development cycle, which I haven't talked about at length.

Sounds fun.

Yeah.

So this is attention.

It's like how do we be um

scientific at the different stages.

So different things we can run ablations on and some of them like mixing are just like

Here is our process.

It's like we did this.

It's like it's very much unacademic.

But in like the RL side, it's like, oh, we can compare different initializations for the value model, you can compare different regularization, these specific hyperparameters.

So there's definitely both of that, which is like you need people that can operate in this chaos, which is just like intuitively like where do we go?

But you also need people who are very scientific as like, yes, no, does this work based on X clear results?

So it kinda it has the full spectrum there.

And I

I mean I agree.

Showing 681–700 of 1,013 · page 35 of 51