Yann LeCun
speaker
384 appearances
5 recordings
4 series
first heard Mar 2024
last heard 20 Jun
Yann LeCun’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 4 in all, peaking in Jan 2026 with 2.
Appearances
So that's still a gradient-based... It's gradient-based inference. So now you have a representation of the answer in abstract space. Now you can turn it into text. And the cool thing about this is that the representation now can be optimized through gradient descent, but also is independent of the language in which you're going to express the answer.
sensory information right okay but this can can this do something like reasoning which is what we're talking about well not really in a only in a very simple way i mean basically you can think of those things that's doing the kind of optimization i was i was talking about except they optimize in the discrete space which is the space of possible sequences of of tokens
And they do this optimization in a horribly inefficient way, which is generate a lot of hypotheses and then select the best ones. And that's incredibly wasteful in terms of computation. Because you basically have to run your LLM for every possible generated sequence. And it's incredibly wasteful.
So it's much better to do an optimization in continuous space where you can do gradient descent as opposed to like generate tons of things and then select the best. You just iteratively refine your answer to go towards the best, right? That's much more efficient. But you can only do this in continuous spaces with differentiable functions.
Right. So then we're asking the question of, conceptually, how do you train an energy-based model? So an energy-based model is a function with a scalar output, just a number. You give it two inputs, x and y, and it tells you whether y is compatible with x or not. x you observe. Let's say it's a prompt, an image, a video, whatever.
And y is a proposal for an answer, a continuation of the video, you know, whatever. And it tells you whether y is compatible with x. And the way it tells you that y is compatible with x is that the output of that function would be zero if y is compatible with x. It would be a positive number, non-zero, if y is not compatible with x.
Okay, how do you train a system like this at a completely general level is you show it pairs of X and Ys that are compatible, a question and the corresponding answer, and you train the parameters of the big neural net inside to produce zero.
Okay, now that doesn't completely work because the system might decide, well, I'm just going to say zero for everything. So now you have to have a process to make sure that for a wrong y, the energy would be larger than zero. And there you have two options. One is contrastive method.
So contrastive method is you show an x and a bad y, and you tell the system, well, that's, you know, give a high energy to this, like push up the energy, right? Change the weights in the neural net that computes the energy so that it goes up. So that's contrasting methods.
The problem with this is if the space of Y is large, the number of such contrasting samples you're going to have to show is gigantic. But people do this. They do this when you train a system with RLHF. Basically what you're training is what's called a reward model, which is basically an objective function that tells you whether an answer is good or bad. And that's basically exactly what
what this is. So we already do this to some extent. We're just not using it for inference, we're just using it for training. There is another set of methods which are non-contrastive, and I prefer those. And those non-contrastive methods basically say, okay, the energy function needs to have low energy on pairs of x, y's that are compatible, that come from your training set.
How do you make sure that the energy is going to be higher everywhere else? And the way you do this is by having a regularizer, a criterion, a term in your cost function that basically minimizes the volume of space that can take low energy. And the precise way to do this is all kinds of different specific ways to do this, depending on the architecture. But that's the basic principle.
So that if you push down the energy function for particular regions in the XY space, it will automatically go up in other places because there's only a limited volume of space that can take low energy by the construction of the system or by the regularizing function.
Yeah, so you can do this with language directly by just, you know, x is a text and y is a continuation of that text. Yes. Or x is a question, y is the answer.
Well, no, it depends on how the internal structure of the system is built. If the internal structure of the system is built in such a way that inside of the system there is a latent variable, let's call it z, that... you can manipulate so as to minimize the output energy. Then that Z can be viewed as a representation of a good answer that you can translate into a Y that is a good answer.
Very similar way, but you have to have this way of preventing collapse, of ensuring that there is high energy for things you don't train it on. And currently it's very implicit in LLM. It's done in a way that people don't realize is being done, but it is being done. It's due to the fact that when you give a high probability to a word,
automatically you give low probability to other words because you only have a finite amount of probability to go around right there, to sum to one.
So when you minimize the cross entropy or whatever, when you train your LLM to predict the next word, you're increasing the probability your system will give to the correct word, but you're also decreasing the probability it will give to the incorrect words. Now,
Indirectly, that gives a high probability to sequences of words that are good and low probability to sequences of words that are bad, but it's very indirect. It's not obvious why this actually works at all, because you're not doing it on a joint probability of all the symbols in a sequence.
You're just doing it to factorize that probability in terms of conditional probabilities over successive tokens.
Showing 201–220 of 384 · page 11 of 20
← Previous
Next →