Andrej Karpathy

And so it's the universal interface that can basically command the digital infrastructure we've built up for ourselves.

And so it feels like a very powerful interface to command and to build on top of.

Now, to your question as to what I learned from that, it's interesting because the world of bits was basically too early, I think, at OpenAI at the time.

This is around 2015 or so, and the zeitgeist at that time was very different in AI from the zeitgeist today.

At the time, everyone was super excited about reinforcement learning from scratch.

This is the time of the Atari paper, where neural networks were playing Atari games and beating humans in some cases, AlphaGo and so on.

So everyone was very excited about training neural networks from scratch using reinforcement learning directly.

It turns out that reinforcement learning is an extremely inefficient way of training neural networks, because you're taking all these actions and all these observations, and you get some sparse rewards once in a while.

So you do all this stuff based on all these inputs, and once in a while you're told, you did a good thing, you did a bad thing.

And it's just an extremely hard problem.

You can't learn from that.

You can burn a forest, and you can sort of brute force through it.

And we saw that, I think, with Go and Dota and so on.

And it does work.

but it's extremely inefficient and not how you want to approach problems, practically speaking.

And so that's the approach that at the time we also took to World of Bits.

We would have an agent initialize randomly.

So with keyboard mash and mouse mash and try to make a booking.

And it's just like revealed the insanity of that approach very quickly, where you have to stumble by the correct booking in order to get a reward of you did it correctly.

And you're never going to stumble by it by chance at random.