Andrew Ilyas

I think most of the analysis was done on a smaller version of Claude that they then had to extrapolate to a larger version.

And so there are all of these things that are, I think, fundamentally right now, just like engineering, clever methods.

But I think as we sort of scale up and get these methods to compete with the size and scale of models and data sets, I think there will be a lot of cool stuff we can learn.

Yeah, I think they're just fundamentally different questions that you ask at each scale.

I think by choosing to be in academia, perhaps I have more freedom to do things at small scale, but I'm not going to be able to run the experiments that OpenAI are running.

And to some extent, that is a barrier between academia and industry.

But I think that the way that academia has to deal with that is by sort of prioritizing different questions.

almost in the same way that we have Boeing and then we have aerospace engineering professors that aren't building commercial airliners.

They're just fundamentally different questions that emerge at each scale.

And so I think what I hope academia can do is really try to understand, are there things that hold across scales that we can then verify at very large scale?

But I think a lot of the problems that are currently in machine learning of like, you know, the things you see in ChatGPT, like your bias, hallucination, you know, incorrect facts, all of these things are problems we can study at small scale and try to make predictions about.

Yeah, absolutely.

I mean, I totally agree.

And I think a lot of that work is really, really cool.

And I think in general, I mean, I think this is definitely a thing in design.

They say, like, constraints breed good design.

Because, like, you know, there's always the thing you could do of just, like, what happens if I scale this method up, you know, a thousand X?

And I'm sure it does better than it does now.

And I think there are really interesting questions.

I truly think there are really interesting questions that can only be studied at