Emad Mostaque

What's the right compute?

Like our image model, we have over 120 different clusters of images.

Only like nine are used, like 95% of the time.

All the rest of the data is just bunkum.

What does that look like for a language model?

Like, do you need to train it on all of those auto-generated transcripts of, like, Spider-Man pulling out someone's tooth on YouTube and all these weird videos?

There's a whole subculture of generated videos where you have, like, Spider-Man and SpongeBob SquarePants and Mickey Mouse, like, having a fight and stuff like that.

I got to find these corners.

It's a deep, dark area of YouTube.

You don't want to go there, man.

Yes, or like a group of humans coming together, there's suddenly a race condition where it just goes.

It's not trying to do something bad.

The humans don't want to do something bad, but it happens.

Just like the example I always give is YouTube optimized for engagement, which then optimized for extreme content, which is optimized for ISIS.

Nobody in YouTube wanted ISIS to do well.

All of a sudden it did, because that's what the algorithm was optimized for.

And so once you start getting agentic AI that you let loose on the internet and they can make decisions according to its reward function, you could get some weird stuff happening.

Agentic AI is AI that can go and pay a bill.

It can go on the internet, can search more stuff.

It comes back like little agents.