Trenton Bricken

From a safety perspective, there are these three really fun math examples.

So in one of them, you ask the model to do square root of 64, and it does it.

And you can look at the circuit for it and verify that it actually can perform this square root.

And in another example, it will add two numbers, and you can see that it has these really cool lookup table features that will do the computation.

for like, the example's 59 plus 36.

So it'll do the five plus nine and know that it's this modulo operation.

And then it will also at the same time do this fuzzy lookup of like, okay, I know one number is a 30 and one's a 50, so it's gonna be roughly 80.

And then it will combine the two, right?

Okay, so with the square root 64, it's the same thing.

You can see every single part of the computation and that it's doing it.

And the model tells you what it's doing.

It has its scratch pad and it goes through it.

And you can be like, yep, okay, you're telling the truth.

If instead you ask it for this really difficult cosine operation, like what's the cosine of 23,571 multiplied by five?

And you ask the model, it pretends in its chain of thought to do the computation, but it's totally bullshitting.

And it gets the answer wrong.

And when you look at the circuit,

it's totally meaningless.

It's clearly not doing any of the right operations.

And then in the final case, you can ask it the same hard cosine question, and you say, I think the answer's 4, but I'm not sure.