Yann LeCun
speaker
384 appearances
5 recordings
4 series
first heard Mar 2024
last heard 20 Jun
Yann LeCun’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 4 in all, peaking in Jan 2026 with 2.
Appearances
And the reason you can tell it's primitive is because the amount of computation that is spent per token produced is constant. So if you ask a question and that question has an answer in a given number of token, the amount of computation devoted to computing that answer can be exactly estimated.
It's like, you know, it's the size of the prediction network, you know, with its 36 layers or 92 layers or whatever it is, multiplied by number of tokens, that's it. And so essentially it doesn't matter if the question being asked
is simple to answer, complicated to answer, impossible to answer because it's undecidable or something, the amount of computation the system will be able to devote to the answer is constant, or is proportional to the number of tokens produced in the answer, right? This is not the way we work.
The way we reason is that when we're faced with a complex problem or a complex question, we spend more time trying to solve it and answer it, right? Because it's more difficult.
Okay, whether it's difficult or not, the near future will say, because a lot of people are working on reasoning and planning abilities for dialogue systems. I mean, even if we restrict ourselves to language, just having the ability to plan your answer before you answer in terms that are not necessarily linked with the language you're going to use to produce the answer.
So this idea of this mental model that allows you to plan what you're going to say before you say it. That is very important. I think there's going to be a lot of systems over the next few years that are going to have this capability. But the blueprint of those systems would be extremely different from autoregressive LLMs. So
it's the same difference as the difference between what psychologists call system one and system two in humans, right? So system one is the type of tasks that you can accomplish without like deliberately, consciously think about how you do them. You just do them, you've done them enough that you can just do it subconsciously, right? Without thinking about them. If you're an experienced driver,
You can drive without really thinking about it, and you can talk to someone at the same time or listen to the radio, right? If you are a very experienced chess player, you can play against a non-experienced chess player without really thinking either. You just recognize the pattern and you play. Right? That's system one.
So all the things that you do instinctively without really having to deliberately plan and think about it. And then there is all the tasks where you need to plan. So if you are... and not to an experienced chess player, or you are experienced when you play against another experienced chess player. You think about all kinds of options, right? You think about it for a while, right?
You're much better if you have time to think about it than you are if you play Blitz with limited time. So this type of deliberate planning, which uses your internal world model, that's system two. This is what LLMs currently cannot do. So how do we get them to do this? How do we build a system that can do this kind of planning or reasoning that devotes
more resources to complex problems than to simple problems. And it's not going to be autoregressive prediction of tokens. It's going to be more something akin to inference of latent variables in what used to be called probabilistic models or graphical models and things of that type. So basically, the principle is like this. The prompt is like observed variables.
And what the model does is that it can measure to what extent an answer is a good answer for a prompt. So think of it as some gigantic neural net, but it's got only one output. And that output is a scalar number, which is, let's say, zero if the answer is a good answer for the question, and a large number if the answer is not a good answer for the question. Imagine you had this model.
If you had such a model, you could use it to produce good answers. The way you would do is produce the prompt and then search through the space of possible answers for one that minimizes that number. That's called an energy-based model.
Well, so really what you need to do would be to not search over possible strings of text that minimize that energy. But what you would do is do this in abstract representation space. So in sort of the space of abstract thoughts, you would elaborate a thought, right, using this process of minimizing the output of your model, okay, which is just a scalar. It's an optimization process.
So now the way the system produces its answer is through optimization, by minimizing an objective function, basically. And we're talking about inference, we're not talking about training. The system has been trained already. So now we have an abstract representation of the thought of the answer, representation of the answer. We feed that to, basically, an autoregressive decoder,
which can be very simple, that turns this into a text that expresses this thought. So that, in my opinion, is the blueprint of future dialogue systems. They will think about their answer, plan their answer by optimization before turning it into text. And that is Turing-complete.
The space of representations. It goes abstract representation. Abstract representation. So you have an abstract representation inside the system. You have a prompt. The prompt goes through an encoder, produces a representation, perhaps goes through a predictor that predicts a representation of the answer, of the proper answer.
But that representation may not be a good answer because there might be some complicated reasoning you need to do, right? So then you have another process that takes the representation of the answers and modifies it so as to minimize a cost function that measures to what extent the answer is a good answer for the question.
Now, we sort of ignore the fact for, I mean, the issue for a moment of how you train that system to measure whether an answer is a good answer for a question.
It's an optimization process. You can do this if the entire system is differentiable, that scalar output is the result of running through some neural net, running the answer, the representation of the answer through some neural net. Then by gradient descent, by back-propagating gradients, you can figure out how to modify the representation of the answer so as to minimize that.
Showing 181–200 of 384 · page 10 of 20
← Previous
Next →