Jaden Schaefer

So for many decades, these kind of neural networks were basically ignored.

But that all stopped after three main things happened.

So first, of course, data exploded.

You have the internet, you have smartphones, you have social media, so so much data is being created.

And suddenly we have all of this data specifically about languages and images and behavior and everything.

So all of this data.

And then second...

compute got super, super cheap and also powerful.

So the GPUs that were, you know, originally built for gaming, they turned out to be really perfect for training neural networks.

And I mean, I would even say go so far as to say, like a lot of the hardware that was built for crypto mining.

And then when the crypto winter came, that just kind of perfectly pivoted into AI.

And we had like all of this infrastructure built out that had we not been through that, we wouldn't have been able to kind of uptick training AI models as fast as we did.

So

Those that all helped.

And I think the last thing that really helped was that researchers figured out some better techniques for training deep neural networks.

And this is like this is kind of where this deep learning comes in.

It's basically the idea that you stack a whole bunch of layers of neural networks to learn harder and more complex patterns.

And basically by kind of adding all of that, the data, the compute and that new strategy, everything changed.

So in the early 2010s, deep learning started to crush a lot of benchmarks.

It's also hilarious to talk about crushing benchmarks in 2010 because it's definitely different than what we have today.