Andy Ellis

We're gonna assume they're good.

Oh, see, I would assume they're not good.

Well, you're assuming that you're trying to create good metrics.

Because one of the situations is you know you got better and the metrics don't show it.

Therefore, it's not a good metric.

If you don't understand how the metric got better, more likely than not, it's not a good metric.

Like I once bought from a security vendor a tool that sat on the network and detected lateral movement so that we could find things coming in and our IT team.

Well, then now we're just blaming the metric and so it doesn't really matter.

Yeah, no, I'm totally going to blame the metrics.

I think this is a metrics problem.

I think that you are worse off if your metrics tell you you're getting better, but you have no idea how you're getting better because you now are lulled into a sense of false confidence.

You didn't actually get better.

Your metric just tells you you are.

It's possible that it could be, but let's just go with like what we've normally seen.

Like I've seen reports from tools that said, look, everything's great, but it turns out they weren't looking for the right things anymore.

They were based on old technology.

So our IT team thought everything was golden.

Everything was not golden.

I'm saying if your metrics claim you're getting better and you have no idea how you're getting better, you're probably not actually getting better.

And if you know you're getting better and your metrics don't show it, it's an opportunity for you to figure out what your metrics ought to be.