Jesse Zhang

And that's actually pretty difficult.

And I do think that is...

broadly true for anyone that's trying to build in this style of company.

If you're going to be replacing human labor, you need to know what good human labor is.

So what we found is like, okay, well, can someone just tell us what are the answers to all these questions?

And most people don't actually know because these are large, complex organizations.

No one is like the person where like, hey, I know how to answer all these questions.

And so you have to design a process where it's very easy to extract these answers from all the people that do know.

So maybe it's all the CX leaders or people lead different areas of the product.

And so you have to get them all together and get them to align on like, OK, here's what the eval is, essentially.

If you can do that well, then it makes everything a lot easier because now you're just building, building, building.

You have this quantifiable score that's like, hey, here's how well AI is performing.

And then once you're done building and the score is high, then you can go live.

Yeah, yeah, yeah.

That's an interesting way to think about it.

And it doesn't have the reinforcement learning in the pure sense of training a model.

It can just be reinforcement learning and making the agent improve.

And that could be compiling more evals.

That could be compiling just like more guardrails, guidelines around what it can and can't do.

Very fast.