Yann LeCun

And that's not there in LLMs. LLMs are purely trained from text. So then the other statement you made, I would not agree with the fact that implicit in all languages in the world is the underlying reality. There's a lot about underlying reality which is not expressed in language.

And that's not there in LLMs. LLMs are purely trained from text. So then the other statement you made, I would not agree with the fact that implicit in all languages in the world is the underlying reality. There's a lot about underlying reality which is not expressed in language.

You don't need to communicate the stuff that is common.

You don't need to communicate the stuff that is common.

You don't need to communicate the stuff that is common.

I just think that most of the information of this type that we have accumulated when we were babies is just not present in text, in any description, essentially.

I just think that most of the information of this type that we have accumulated when we were babies is just not present in text, in any description, essentially.

I just think that most of the information of this type that we have accumulated when we were babies is just not present in text, in any description, essentially.

I mean, that's the 16,000 hours of wake time of a four-year-old and 10 to the 15 bytes, you know, going through vision, just vision, right? There is a similar bandwidth, you know, of touch and a little less through audio. And then text doesn't, language doesn't come in until like, you know, a year in life. And by the time you are nine years old, you've learned everything.

I mean, that's the 16,000 hours of wake time of a four-year-old and 10 to the 15 bytes, you know, going through vision, just vision, right? There is a similar bandwidth, you know, of touch and a little less through audio. And then text doesn't, language doesn't come in until like, you know, a year in life. And by the time you are nine years old, you've learned everything.

I mean, that's the 16,000 hours of wake time of a four-year-old and 10 to the 15 bytes, you know, going through vision, just vision, right? There is a similar bandwidth, you know, of touch and a little less through audio. And then text doesn't, language doesn't come in until like, you know, a year in life. And by the time you are nine years old, you've learned everything.

about gravity, you know, about inertia, you know, about gravity, you know, the stability, you know, you know, about the distinction between animate and inanimate objects, you know, by 18 months, you know, about like why people want to do things and you help them if they can't, you know, I mean, there's a lot of things that you learn mostly by observation, really not even through interaction.

about gravity, you know, about inertia, you know, about gravity, you know, the stability, you know, you know, about the distinction between animate and inanimate objects, you know, by 18 months, you know, about like why people want to do things and you help them if they can't, you know, I mean, there's a lot of things that you learn mostly by observation, really not even through interaction.

about gravity, you know, about inertia, you know, about gravity, you know, the stability, you know, you know, about the distinction between animate and inanimate objects, you know, by 18 months, you know, about like why people want to do things and you help them if they can't, you know, I mean, there's a lot of things that you learn mostly by observation, really not even through interaction.

In the first few months of life, babies don't really have any influence on the world. They can only observe, right? And you accumulate a gigantic amount of knowledge just from that. So that's what we're missing from current AI systems.

In the first few months of life, babies don't really have any influence on the world. They can only observe, right? And you accumulate a gigantic amount of knowledge just from that. So that's what we're missing from current AI systems.

In the first few months of life, babies don't really have any influence on the world. They can only observe, right? And you accumulate a gigantic amount of knowledge just from that. So that's what we're missing from current AI systems.

Right. So because of the autoregressive prediction, every time an LLM produces a token or a word, there is some level of probability for that word to take you out of the set of reasonable answers. And if you assume, which is a very strong assumption, that the probability of such error is that those errors are independent across a sequence of tokens being produced.

Right. So because of the autoregressive prediction, every time an LLM produces a token or a word, there is some level of probability for that word to take you out of the set of reasonable answers. And if you assume, which is a very strong assumption, that the probability of such error is that those errors are independent across a sequence of tokens being produced.

Right. So because of the autoregressive prediction, every time an LLM produces a token or a word, there is some level of probability for that word to take you out of the set of reasonable answers. And if you assume, which is a very strong assumption, that the probability of such error is that those errors are independent across a sequence of tokens being produced.