Peter McCrory

Sort of the mismeasured consumer surplus.

I mean, that's also been a lesson of recent information technologies, the advent of Google, the ability to get information at your fingertips didn't show up in GDP.

It's sort of for most of us, it's free in some sense.

And

Yeah, I mean, I think this is a really interesting point that the greatest value might ultimately be unmeasured, at least from the standpoint of GDP.

I'm sympathetic to the critique of GDP as a measure of sort of technological progress and prosperity, but it is also the case that over time and across countries, so many other

choose your favorite measure of human prosperity, it tends to correlate very strongly with GDP.

So it might not be the case that it's capturing everything, but it might be pointing us in the right direction.

I'm a bit familiar with the empirical-based large-scale inference techniques that can allow you to sift out some signals when you do large-scale testing in this way.

But I totally agree with the point in general that

We may be overwhelmed, and this is not just in the scientific domain, we may be overwhelmed, an immensity of information that is very challenging to process, and you might be able to use large language models to help you process that information as well.

One of the, I'm like bouncing around in my mind, but one of the things that you said reminded me of, in the paper, we sort of do this exercise where we wanna like come up with an effective AI coverage measure, takes into account task reliability,

How much time are you spending on different tasks?

One of the examples that we call out where this seems to matter is for microbiologists.

So if you just count up the tasks that microbiologists do, it looks like they have about half of their tasks are things that Claude can and is actually being used for.

But when you focus on the most time-intensive tasks for that role, like hands-on research with specialized lab equipment, large language models are not at the point where

we are able to sort of automate that process.

And maybe we need more general purpose robotics that can interact with this intelligence.

But until we kind of figure out that bottleneck, we will be constrained in how much we can sort of achieve, even if we have the most brilliant microbiologists embedded in Claude's capabilities.

Yeah, I mean, I guess it might be leveraging the burden that is placed on one person as you sort of are taking on more responsibility, to your point, especially in the legal domain.