Andrew Ilyas

But I think this idea that you have these two, what I call it, two really nice unique properties.

One is this universal interpretability of the embeddings.

And the second is the fact that not only do the individual coordinates mean something, but the vector as a whole, you can treat as a predictor of how the model is going to behave.

These two things together, I think, allow you to basically extract a lot of cool insights about, again, learning algorithms.

Yeah, absolutely.

So you can use this to analyze a specific learning algorithm and try to understand what its failure modes are and things like that.

When we compare it to two different learning algorithms or two different classes of machine learning models, what we're doing is basically looking for systematic differences in these data model vectors and using those to try to understand, OK, well, what

for what subpopulations does one model class or one learning algorithm work substantially differently from the other?

Yeah, I think the big next steps, I'm obviously still working on this space.

I think I'd highlight three big next steps.

One is just making this more efficient and more performant.

The data model estimator in its original form requires you to train, I don't know, tens of thousands of machine learning models on random data sets, which is not ideal.

And we have some follow-up work called track making this faster, but I think we really haven't hit the limit.

There's still a lot of room for improvement.

And right now, you know, we're working on better, faster versions of this data modeling problem that I think are going to be really exciting.

The second thing is finding applications specifically sort of in

data hungry regimes that we maybe don't normally think about sort of like outside of vision or even outside of language modeling.

But thinking, for example, about like thinking about robotics, thinking about like, you know, scientific measurement type stuff where really like, you know, pointing your telescope somewhere and taking a picture is like a very expensive thing and you want to know exactly where you should do it.

So I think like applying it to those outside domains that we don't usually think about something else that's really exciting.

Yeah, it's a really good question, and I think it needs a little thought.