Grant Harvey

smaller sets of data to improve the weightings.

Like for clarity, by the way, I am not an ML scientist.

So I'm going to say some things and they'll be listening as being like, this guy's a charlatan.

So just for clarity, before I get flamed, I'm giving some vibes here rather than the exact science.

And then I think the evaluation speed is so key.

Like there's all these different models and in lots of ways, like you could go, if you go to

chat GPT, Claude, Gemini, and you ask it a reasonably simple question, it's going to give you the right answer pretty much.

Or if you give it a complex, subjective question, it's going to give you some different answers.

But how the hell do you decide which is better?

That's where evaluation is so critical.

I think Andre Carpenter- Someone has to read-

Well, Henri Carpenter had a tweet about six months ago where he said, like, the entire AI is just an evaluation problem now.

Because it's like the models are getting so complex that understanding what better means, they're not linearly better.

You might get an improvement in, say, let's imagine you get an improvement in creativity, but that also makes it more verbose and more likely to hallucinate.

You might be more creative, which is great.

I mean, AI is not magic.

It's one of the biggest problems.

People think AI is not magic.

AI is just better predictive text.

Like it's very good predictive text.