Nathan Lambert
speaker
1,013 appearances
1 recordings
1 series
first heard Nov 2024
last heard Nov 2024
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
if you have the weights, you can get the model to like generate ish prompts from its distribution, and then you can check those on test sets.
Um
Which is really funny 'cause Magpie is meant as kind of like a training architecture for like generating new data.
But when you then realize that that might be what you need to do for a valid, I told the author and he's like, Oh, oh, cool.
Another use case for my paper.
Which is really funny, but I do think that whole ecosystem is gonna continue to grow a lot because it's you need it to have
some source of truth, I think.
Avals are only growing in value and cost, and you need to be able to audit the models to assess this a across industry.
So I dunno there's gonna be some mix of new startups, academic research, regulation on decontamination.
Like there's it the whole thing is kind of a fair game.
Mostly the same.
You have to add new tok it's it's the same, but you add in new tokens.
So you add like the tokens to help constrain the model, which is like user
Or some other like EOS token, you add these things in in a specific format.
So you're
You have to learn a little bit of new tokens, but otherwise it's exactly the same.
Yeah, it's almost like for simple things, yeah, I don't think you need a lot of SFT data. 'Cause you're just like manipulating the weights to be more focused on this type of formatting.
Like you need to do enough so that this is the way the model outputs.
But for specific tasks like math, I think you can make a lot bigger gains, which is just like the model has not seen a lot of this and you're really trying to like
do the same thing that you do during pre-training, which is like teach the model to do chain of thought, math.
Showing 341–360 of 1,013 · page 18 of 51
← Previous
Next →