Zach Lipton

And so I say is at any given point in time, like those areas that we talk about, the ones that we have almost like made into a bit of a science, like we put our finger on it, we can measure them very cleanly.

Those are actually the easier ones.

Like these are the easiest aspects.

You know, again, I was giving you like the metaphor of like imagine trying to assess your journalists in a single number.

I think a similar one would be a software developer.

How good is your software developer?

And what you find is like you say, well, how many lines of code are they producing?

You know, how many commits are they making?

What fraction of their commits have to be rolled back?

What if I were to look at their teams, you know, the linear tickets that were assigned to them, you know, how many were closed and by what time?

And you could construct a whole bunch of metrics

And then you say, well, which one of those is the hardest to work on?

And in some ways it's like all the things that were easy to measure, that you knew how to measure, that you had a clean word for, that you had a great statistic for, those are actually the easy ones.

And the hardest thing is when you make a change to a model and then, so we have a cascade here.

We have this very built out machinery of automated metrics, but we don't just say then, okay, the machines like it, we're gonna deploy it.

We then have an internal review with a set of actual trained clinicians.

And then we do a blinded review with a set of external clinicians.

And it's only then that we start this cascade.

And the hardest thing of all is when you're in a situation where all the metrics look great and some of us, they don't like it.

And the hardest thing is the one you don't have a word for, that you don't have your finger on.