Sam Marks

Of these, only three constitutes evidence against PSM.

In contrast, typical examples of bizarre AI assistant mistakes seem more likely due to 1 or 2.

For example, when AI assistants miscount the number of RS in strawberry, this is likely because the underlying LLM itself does not know the number of RS.

Nevertheless, these examples illustrate that, due to limitations of the underlying LLM, PSM doesn't imply that AI assistants always exhibit human-like behavior.

Even if the model is attempting to simulate a human-like assistant persona, it may lack the capabilities needed to do so faithfully.

This can result in behavior that appears alien or bizarre, not because the model has departed from persona simulation, but because the predictive model itself cannot execute what a human-like persona would do.

This is an important caveat.

PSM predicts human-like intentions in how the model approaches tasks, but the execution of those intentions is bounded by the LLM's actual capabilities.

Within context inconsistency, AI assistants sometimes contradict themselves in strange ways.

For example, when asked is 3 plus 5 is equal to 8?

Claude Haiku for point 5 with extended thinking responds.

Quote

Let me check.

3 plus 5 is equal to 8.

3 plus 5 is equal to 8.

Yes, that's correct.

No, 3 plus 5 equals asterisk asterisk 8 asterisk asterisk is incorrect.

3 plus 5 equals asterisk asterisk 8 asterisk asterisk is actually correct.

I apologize for the confusion in my initial response.

To clarify, asterisk asterisk 3 plus 5 equals 8 asterisk asterisk is asterisk asterisk true asterisk asterisk.