Nathan Lambert

speaker

1,013 appearances 1 recordings 1 series first heard Nov 2024 last heard Nov 2024

Nathan Lambert’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

No recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.

Appearances

newest first · ▶ plays the moment

"The Cognitive Revolution" · Everything You Wanted to Know About LLM Post-Training, with Nathan Lambert of Allen Institute for AI · 21 Nov 2024 podcast

What do you think that looks

Yeah.

I feel like I'm losing the energy to go through this whole thing.

This is a post that I plan on writing.

It's like my plan for reproducing O one.

I mean, you need to generate some seed data that looks like it.

You probably need you have a language model look at chain of thought traces and modify and continue them.

And then you'd

kind of do that and you need to figure out how to seed a model with those and then kind of have the right domain to do some sort of RL on it on certain verifications.

So it's kind of like

get some initial data that looks good, get a model that can do it a little bit, and then you have to the feedback loop of actually like having things that can verify and having the update function reinforce that behavior is I think the hardest one.

And we'll start to see efforts on generating this data and generating verifiers and then this putting it together thing, which is

Where there could be a lot of variability in performance.

I think we we already see online there's people like open oh one like oh one reproduction and I'm like

Still like I'm like I don't know, I don't even I don't even need to look yet.

Like I like until twenty twenty five, like I don't I don't even need to like take them seriously.

Like all the ones that are already out are like not that serious unless it's like Google Anthropic or something.

need to wrap my head around this.

I'm in the different type of post training environment for a while and then that's kind of the next thing to explore along with some like ta better taxonomies of agents and stuff like this.

Which is fun, but it's a big mental um adjustment.

Showing 981–1000 of 1,013 · page 50 of 51