Grant Harvey

Well, I guess I should clarify.

Let's say if we go off of the previous scaling paradigm, which is just size and data, right?

Compute and data.

If you were to scale up to the next size up of compute and data, if that were even possible...

Would that model not think for very long?

And then would that then make it, it makes me think that that model's thinking traces would be even harder to interpret because it's, you know, following the logic of what you've proven so far.

As it scales.

Yeah, it's a bigger model and it's thinking for less time to get to the right answer.

So speaking of open windows into the mind, you know, we're looking at the thoughts here.

You also published a paper recently on weight sparse transformers where you prune the model to make its internal circuits easier to see.

How does like, I guess, mechanistic interpretability and chain of thought monitor ability work?

Right.

You at least contributed to the system, the model card for it, right?

Yeah.

Yeah.

We have a question just that's kind of for fun, which is, how does research work at OpenAI?

Like, do you all pitch to Sam and then he gives you the thumbs up or Jacob or whoever's in charge of the science there?

Or do you get it assigned to you?

Like, how do you decide what your next paper is going to be?

Do you guys have to fight for compute?