Jeremiah

The AI certainly isn't answering your math question by thinking something like, hmm, she used the number 3, which has the tokens TH and RE, and I know that there's an 8.2% chance that RE is often seen somewhere around the token X, so the answer must be 6.

How would that even work?

Instead, consider your own evolution.

On the outermost level, humans were designed by a process optimizing for survival, sex, and reproduction.

The humans that survived were those that had sex and reproduced.

Everything about humans is downstream of what helped with sex and reproduction.

But that doesn't mean that any particular thought that you think involves reproduction or sex.

If you're doing a math problem, you won't think, hmm, how can I have sex with the number three?

You're not even thinking, in order to reproduce, I need to survive.

To survive, I need money.

To get money, I need a good job.

And to get a good job, I need good grades.

And to get good grades, I need to get the answer to this math problem.

Therefore, the answer is 76.

You're just doing good, normal math.

The evolutionary process that designed the learning algorithms that power your brain was thinking about, all in quotes, survival and sex and reproduction, but you may never consider those things at all in the course of any given task.

CF, organisms are adaption executors, not fitness maximizers.

Link in post.

Which does a good job hammering in the point that we run algorithms designed by the evolutionary imperative to maximize survival and reproduction.

rather than considering survival and reproduction explicitly in our decisions.