Dylan Patel

Go figure it out.

And then they spend a whole summer doing something you could have done in like three days.

But like, great, they learned a shit ton.

These models need to go through that progression.

And so when I think about reasoning RL, it's a lot about how the human psyche and intelligence works.

There's a caution of like trying to make it too much like humans because it's not the fundamental substrate is not like humans.

The processing is not like humans.

Our brain is very different from how these ALUs on a chip works.

Like the scaling of these things is very different.

The raw speed, the amount of words they can, everything is so different.

But at the same time, it's important to like reckon back to what actually makes people, you know, smart.

The magic of transformers was attention, i.e.

I calculate everything in my context length.

I calculate the attention to each other.

Basically in a vector space like king, queen, there's these vectors, there's like dozens of vectors for each number.

And king and queen are actually exactly the same on a ton of stuff, but then it's the opposite on one number because one's a male, one's a female.

And then that will have a lot of other like ramifications throughout other literary stuff.

Like what adjectives do you put with a male of, you know, these vectors?

Oh, it's like

regal and powerful and could be ruthless, whereas a queen could be dignitary or whatever.