Joel Hron

And so that's really, you know, what what some of our main focus has been for the last, you know, nine months or so.

Yeah.

I mean, we've certainly heard that.

I mean, some of like you mentioned, you know, the thousand plus customers we've been testing this with.

And I think the first thing we've heard, which is maybe the obvious thing is like, oh, you know, this took something that would have been a 10 hour research project into 10 minutes.

And that's all well and good, but I think the other anecdotes that we've heard from some of the customers is that this surfaced new arguments or new ways of looking at the law that hadn't been surfaced before.

And I think that's one of the most profound and compelling opportunities here.

Yeah, even if I did have an example, I probably would butcher the legal interpretation of what it is.

But yeah, we have some good anecdotes from customers where they've been able to use this to, again, help think through what are good arguments for this case and how can I create new arguments that maybe I hadn't thought of before.

That's amazing.

And it really speaks to also, I think, the process of building it with experts in the loop.

Like, you know, one of the things early on that we and that's certainly something we have.

We have, I think, 4,500 plus domain experts that like sit with our teams doing this work and helping them.

you know, score answers, you know, do preference judgments, identify good rubrics for correctness or helpfulness or these other things about the answers.

And one of the things early on, we, you know, you sort of craft these gold sets of questions that you want to be able to have the system answer.

And one of the obvious things is like, as a rubric is to say, okay, an answer to this question should reference these cases as source material.

And, you know, you could design a system that like evaluates, okay, did the answer generate references to these cases?

And is that good or bad?

What we found is that like the system, if you give it enough agency, was able to get to the right answer along a number of different paths or trajectories.

Like maybe it would cite to different cases or different sources of secondary law or primary law or otherwise.