Tamay Besiroglu

If you scale up data collection as well, I think it gets even stronger, like real-world data collection by deployment and so on.

But building Shenzhen in a desert, that's a pretty... Like, if you think about the...

Pipeline.

So, so far we have relied – first of all, we're relying on the entire semiconductor supply chain.

That industry depends on tons of inputs and materials and whatever.

It gets from probably tons of random places in the world.

And creating that infrastructure, like doubling or tripling, whatever, that infrastructure –

like the entire thing, that's very hard work, right?

So probably you couldn't even do it even if you just have shens in a letter.

Like that would be even more expensive than that.

And on top of that, so far we have been drawing heavily on the fact that we have built up this huge stock of data

over the past 30 years or something on the internet.

Like, imagine you were trying to train a state-of-the-art model, but you only have, like, 100 billion tokens, right, to train on.

That would be very difficult.

So, in a certain sense, our entire economy...

has produced this huge amount of data on the internet that we are now using to train the models.

It's plausible that in the future when you need to get new competencies added to these systems, the most efficient way to do that will be to try to leverage similar kind of modalities of data, which will also require this like

You would want to deploy the systems broadly because that's going to give you more data.

Maybe you can get where you want to be without that, but it would just be less efficient if you're starting from scratch compared to if you're collecting a lot of data.

I think this is actually a motivation for why labs want their LLMs to be deployed widely because sometimes when you talk to ChatGPT, it's going to give you two responses and it's going to say, well, which one was good?