Andrej Karpathy

I think as neural nets converge to humans, the value of simulation to neural nets will be similar to the value of simulation to humans.

So people use simulation because they can learn something in that kind of a system.

and without having to actually experience it.

No, sorry, simulation, I mean like video games or other forms of simulation for various professionals.

Okay, that's like internal simulation.

Yeah, but that's independent from the use of simulation in the sense of computer games or using simulation for training set creation.

Yeah, that's a different simulation from like Unreal Engine.

That's how I interpreted the question.

Yeah, the graphics, the physics, and the agents that you put into the environment and stuff like that.

I think humans use simulators and they find them useful.

And so computers will use simulators and find them useful.

Yeah, maybe not.

I don't see it as a fundamental, really important part of training neural nets currently.

But I think as neural nets become more and more powerful, I think you will need fewer examples to train additional behaviors.

And simulation is, of course, there's a domain gap in a simulation.

That's not the real world.

It's slightly something different.

But with a powerful enough neural net, the domain gap can be bigger, I think, because the neural net will sort of understand that even though it's not the real world, it has all this high-level structure that I'm supposed to be able to learn from.

A hundred percent.

I just think like at some point you need a massive data set.