Joel Hron

But it would ultimately get to the right answer just in different ways, even if you maybe ran it multiple times.

And so it was an interesting observation.

And I think, you know, really speaks to the need to have like real human experts in the loop evaluating these systems.

Because it is not like a black or white, like binary type answer.

There's a lot of subjectivity in the practice of law, a lot of like domain interpretation in the practice of law that is important to build into the system kind of from the ground up.

Yeah.

that's certainly our intent and you know certainly not an easy thing to do in all cases because you really do want i think to strike a good balance here between uh giving the models the agency to go out and explore these paths um uh because that's really like i think what they're good at and something like like i mentioned the breadcrumb trail things like

You want to give the models agency to follow these paths that may not look obvious at the outset, but might be, you know, elucidating of something new or different.

And so you want to balance that like agency with some structure as well, like some structure about like how good legal process is.

should be done and how to make that also transparent to the user in terms of the answers that are being generated and also the trajectories that the model is following along the way.

And so, yeah, we spend a lot of time finding what that right balance really is between domain orientation and agency on the model side.

I would say I've been super impressed, to be honest with you.

And I don't mean that like blowing smoke or anything like it really is super impressive.

I think the level of quality that the systems are able to deliver.

I think I think.

i think overall like when we when it's not a case where we don't see any errors in any answers though and so this question often comes up is like okay like do i just hit this button and send it to send it to the courthouse like i'm done

And I don't think that's reality.

I don't think it will be reality for for quite a long time.

Yeah, I think I think the models are super impressive and what they're able to do.

But, you know, they really are just, I think, augmentations to really good attorneys at the end of the day rather than like full on placements for them.