Yann LeCun

to learning representations of images by training a system to predict videos, learning intuitive physics by training a system to predict what's going to happen in a video, and tried and tried and failed and failed with generative models, with models that predict pixels. We could not get them to learn good representations of images. We could not get them to learn good representations of videos.

And we tried many times. We published lots of papers on it. They kind of sort of worked, but not really great. It started working. We abandoned this idea of predicting every pixel and basically just doing digital embedding and predicting in representation space. That works.

And we tried many times. We published lots of papers on it. They kind of sort of worked, but not really great. It started working. We abandoned this idea of predicting every pixel and basically just doing digital embedding and predicting in representation space. That works.

And we tried many times. We published lots of papers on it. They kind of sort of worked, but not really great. It started working. We abandoned this idea of predicting every pixel and basically just doing digital embedding and predicting in representation space. That works.

So there's ample evidence that we're not going to be able to learn good representations of the real world using generative model. So I'm telling people, everybody's talking about generative AI. If you're really interested in human-level AI, abandon the idea of generative AI.

So there's ample evidence that we're not going to be able to learn good representations of the real world using generative model. So I'm telling people, everybody's talking about generative AI. If you're really interested in human-level AI, abandon the idea of generative AI.

So there's ample evidence that we're not going to be able to learn good representations of the real world using generative model. So I'm telling people, everybody's talking about generative AI. If you're really interested in human-level AI, abandon the idea of generative AI.

Right. Well, there's a lot of situations that might be difficult for a purely language-based system to know. Like, okay, you can probably learn from reading texts, the entirety of the publicly available texts in the world, that I cannot get from New York to Paris by snapping my fingers. That's not going to work, right? Yes.

Right. Well, there's a lot of situations that might be difficult for a purely language-based system to know. Like, okay, you can probably learn from reading texts, the entirety of the publicly available texts in the world, that I cannot get from New York to Paris by snapping my fingers. That's not going to work, right? Yes.

Right. Well, there's a lot of situations that might be difficult for a purely language-based system to know. Like, okay, you can probably learn from reading texts, the entirety of the publicly available texts in the world, that I cannot get from New York to Paris by snapping my fingers. That's not going to work, right? Yes.

But there's probably more complex scenarios of this type, which an NLM may never have encountered and may not be able to determine whether it's possible or not. So that link from the low level to the high level. The thing is that the high level that language expresses is based on a common experience of the low level, which LLMs currently do not have.

But there's probably more complex scenarios of this type, which an NLM may never have encountered and may not be able to determine whether it's possible or not. So that link from the low level to the high level. The thing is that the high level that language expresses is based on a common experience of the low level, which LLMs currently do not have.

But there's probably more complex scenarios of this type, which an NLM may never have encountered and may not be able to determine whether it's possible or not. So that link from the low level to the high level. The thing is that the high level that language expresses is based on a common experience of the low level, which LLMs currently do not have.

When we talk to each other, we know we have a common experience of the world. A lot of it is similar.

When we talk to each other, we know we have a common experience of the world. A lot of it is similar.

When we talk to each other, we know we have a common experience of the world. A lot of it is similar.

No, I agree with what you just said, which is that to be able to do high-level common sense, to have high-level common sense, you need to have the low-level common sense to build on top of.

No, I agree with what you just said, which is that to be able to do high-level common sense, to have high-level common sense, you need to have the low-level common sense to build on top of.

No, I agree with what you just said, which is that to be able to do high-level common sense, to have high-level common sense, you need to have the low-level common sense to build on top of.

And that's not there in LLMs. LLMs are purely trained from text. So then the other statement you made, I would not agree with the fact that implicit in all languages in the world is the underlying reality. There's a lot about underlying reality which is not expressed in language.