Andrew Ilyas

After that paper came out, there were a couple of follow-up papers, both supporting and building on the estimator, and also a couple of ones in parallel that showed that it wasn't reliable, for example, when networks got too deep or when networks got too wide or under different hyperparameters.

And so out of this was sort of born this field that's, I think, now referred to as data attribution.

And so shameless plug, we gave a tutorial on data attribution earlier at the conference on Monday, and the notes are available online.

But I think that it's a really good way to answer the question you were asking about like, how do all these things fit together?

And I think broadly, you can think of data modeling as sort of formalizing the goal of what you might call like predictive data attribution.

So really, your goal is just you give me a training data set, I predict what your model is going to do.

And so an influence function is one way of doing that.

There are a variety of other ways of doing that.

And I think the point of the data modeling work was to show that this goal was even actually extremely possible.

Because prior to, I think, the data modeling work, there wasn't really convincing evidence that this was even a possible thing to do for deep neural networks.

The Shapley value stuff is also really interesting.

It turns out that there are a lot more connections than we realized, or than I think anyone realized when we were writing the paper.

But broadly, I think the sort of overarching way you can distinguish them is that there's this line of work that's most interested in assigning credit to training examples.

So when you're estimating Shapley values, your actual goal is to sort of understand what is this example's fair share of a model prediction.

for each example in your training set.

So you might have some target image of a dog or something, and you want to figure out how important was each dog in my training set for my model's output on this dog.

Whereas in data modeling, the instantiation we gave in the paper is linear.

And so as a result, the coefficients end up looking a lot like estimating the contribution of different training examples.

But the goal is actually much broader than that.

What we're really trying to build is a generic estimator that just maps from data sets to