Aman Sanger
speaker
350 appearances
2 recordings
1 series
first heard Sep 2024
last heard Oct 2024
Aman Sanger’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
But it has these significant limitations. Even barring capabilities, it does not stream. And that means it's really, really painful to use for things where you want to supervise the output. And instead, you're just waiting for the wall of text to show up. Also, it does feel like the early innings of test time compute and search, where it's just very, very much a v0.
And there's so many things that... like don't feel quite right. And I suspect in parallel to people increasing the amount of pre-training data and the size of the models and pre-training and finding tricks there, you'll now have this other thread of getting search to work better and better.
Yeah, I think most of the additional value from Cursor versus everything else out there is not just integrating the new model fast like 01. It comes from all of the kind of depth that goes into these custom models that you don't realize are working for you in kind of every facet of the product, as well as like the really thoughtful UX with every single feature.
Oh, yeah.
Yeah, I think there are three main kinds of synthetic data. The first is, so what is synthetic data first? So there's normal data, like non-synthetic data, which is just data that's naturally created, i.e. usually it'll be from humans having done things. So from some human process, you get this data. Synthetic data, the first one would be distillation.
So having a language model kind of output tokens or probability distributions over tokens. And then you can train some less capable model on this. This approach is not gonna get you a net, like more capable model than the original one that has produced the tokens.
but it's really useful for if there's some capability you want to elicit from some really expensive high latency model, you can then distill that down into some smaller task specific model. The second kind is when like one direction of the problem is easier than the reverse.
And so a great example of this is bug detection, like we mentioned earlier, where it's a lot easier to introduce reasonable looking bugs than it is to actually detect them. And this is probably the case for humans too. And so what you can do is you can get a model that's not trained in that much data, that's not that smart, to introduce a bunch of bugs in code.
And then you can use that to then train, use a synthetic data to train a model that can be really good at detecting bugs. The last category, I think, is, I guess, the main one that it feels like the big labs are doing for synthetic data, which is... producing text with language models that can then be verified easily.
So like, you know, extreme example of this is if you have a verification system that can detect if language is Shakespeare level and then you have a bunch of monkeys typing in typewriters, like you can eventually get enough training data to train a Shakespeare level language model.
And I mean, this is the case, like very much the case for math where verification is, is, is actually really, really easy for formal, um, formal languages, and then what you can do is you can have an okay model, generate a ton of rollouts, and then choose the ones that you know have actually proved the ground truth theorems and then train that further.
There's similar things you can do for code with leetcode-like problems, where if you have some set of tests that you know correspond to, if something passes these tests, it has actually solved the problem. You can do the same thing where you verify that it's passed the test and then train the model and the output set of passed the tests.
I think it's gonna be a little tricky getting this to work in all domains or just in general. Like having the perfect verifier feels really, really hard to do with just like open-ended miscellaneous tasks you give the model or more like long horizon tasks, even in coding.
Yeah. Verification, it feels like it's best when you know for a fact that it's correct. And then it wouldn't be using a language model to verify. It would be using tests or formal systems. Or running the thing, too.
Yeah, yeah.
I think that that's the category that is, um, most likely to result in like massive gains.
Yeah, so RLHF is when the reward model you use is trained from some labels you've collected from humans giving feedback. I think this works if you have the ability to get a ton of human feedback for this kind of task that you care about.
RLAIF is interesting because you're kind of depending on, like this is actually kind of going to, it's depending on the constraint that verification is actually a decent bit easier than generation. Because it feels like, okay, what are you doing? Are you using this language model to look at the language model outputs and then prove the language model?
But no, it actually may work if the language model has a much easier time verifying some solution than it does generating it. Then you actually could perhaps get this kind of recursive loop. I don't think it's going to look exactly like that. The other thing you could do is... we kind of do is a little bit of a mix of RLA-IF and RLA-HF, where usually the model is actually quite correct.
And this is in the case of CursorTab, picking between two possible generations of what is the better one. And then it just needs a little bit of human nudging with only on the order of 50, 100 examples to kind of align that prior the model has with exactly what you want. It looks different than I think normal RLHF where you're usually training these reward models on tons of examples.
Showing 301–320 of 350 · page 16 of 18
← Previous
Next →