Yann LeCun
👤 PersonAppearances Over Time
Podcast Appearances
But it's not like a completely general robot that would be sophisticated enough to do things like clearing up the dinner table.
But it's not like a completely general robot that would be sophisticated enough to do things like clearing up the dinner table.
Well, I mean, I hope things can work as planned. I mean, again, we've been kind of working on this idea of self-supervised learning from video for 10 years and only made significant progress in the last two or three years.
Well, I mean, I hope things can work as planned. I mean, again, we've been kind of working on this idea of self-supervised learning from video for 10 years and only made significant progress in the last two or three years.
Well, I mean, I hope things can work as planned. I mean, again, we've been kind of working on this idea of self-supervised learning from video for 10 years and only made significant progress in the last two or three years.
So basically, I've listed them already. This idea of how do you train a world model by observation? Mm-hmm. And you don't have to train necessarily on gigantic datasets. I mean, it could turn out to be necessary to actually train on large datasets to have emergent properties like we have with LLMs. But I think there's a lot of good ideas that can be done without necessarily scaling up.
So basically, I've listed them already. This idea of how do you train a world model by observation? Mm-hmm. And you don't have to train necessarily on gigantic datasets. I mean, it could turn out to be necessary to actually train on large datasets to have emergent properties like we have with LLMs. But I think there's a lot of good ideas that can be done without necessarily scaling up.
So basically, I've listed them already. This idea of how do you train a world model by observation? Mm-hmm. And you don't have to train necessarily on gigantic datasets. I mean, it could turn out to be necessary to actually train on large datasets to have emergent properties like we have with LLMs. But I think there's a lot of good ideas that can be done without necessarily scaling up.
Then there is how you do planning with a learned world model. If the world the system evolves in is not the physical world, but it's the world of... Let's say the Internet or some world where an action consists in doing a search in a search engine or interrogating a database or running a simulation or calling a calculator or solving a differential equation.
Then there is how you do planning with a learned world model. If the world the system evolves in is not the physical world, but it's the world of... Let's say the Internet or some world where an action consists in doing a search in a search engine or interrogating a database or running a simulation or calling a calculator or solving a differential equation.
Then there is how you do planning with a learned world model. If the world the system evolves in is not the physical world, but it's the world of... Let's say the Internet or some world where an action consists in doing a search in a search engine or interrogating a database or running a simulation or calling a calculator or solving a differential equation.
How do you get a system to actually plan a sequence of actions to give the solution to a problem? So the question of planning is not just a question of planning physical actions. It could be planning actions to use tools for a dialogue system or for any kind of intelligent system. And there's some work on this, but not a huge amount.
How do you get a system to actually plan a sequence of actions to give the solution to a problem? So the question of planning is not just a question of planning physical actions. It could be planning actions to use tools for a dialogue system or for any kind of intelligent system. And there's some work on this, but not a huge amount.
How do you get a system to actually plan a sequence of actions to give the solution to a problem? So the question of planning is not just a question of planning physical actions. It could be planning actions to use tools for a dialogue system or for any kind of intelligent system. And there's some work on this, but not a huge amount.
Some work at FAIR, one called Toolformer, which was a couple years ago, and some more recent work on planning. But I don't think we have a good solution for any of that. Then there is the question of hierarchical planning. So the example I mentioned of planning a trip from New York to Paris, that's hierarchical. But almost every action that we take involves hierarchical planning in some sense.
Some work at FAIR, one called Toolformer, which was a couple years ago, and some more recent work on planning. But I don't think we have a good solution for any of that. Then there is the question of hierarchical planning. So the example I mentioned of planning a trip from New York to Paris, that's hierarchical. But almost every action that we take involves hierarchical planning in some sense.
Some work at FAIR, one called Toolformer, which was a couple years ago, and some more recent work on planning. But I don't think we have a good solution for any of that. Then there is the question of hierarchical planning. So the example I mentioned of planning a trip from New York to Paris, that's hierarchical. But almost every action that we take involves hierarchical planning in some sense.
And we really have absolutely no idea how to do this. There's zero demonstration of hierarchical planning in AI. where the various levels of representations that are necessary have been learned. We can do two-level hierarchical planning when we design the two levels. For example, you have a dog-like robot. You want it to go from the living room to the kitchen.
And we really have absolutely no idea how to do this. There's zero demonstration of hierarchical planning in AI. where the various levels of representations that are necessary have been learned. We can do two-level hierarchical planning when we design the two levels. For example, you have a dog-like robot. You want it to go from the living room to the kitchen.
And we really have absolutely no idea how to do this. There's zero demonstration of hierarchical planning in AI. where the various levels of representations that are necessary have been learned. We can do two-level hierarchical planning when we design the two levels. For example, you have a dog-like robot. You want it to go from the living room to the kitchen.