Sam Marks

Of these two axes, we think the first one is the most important.

Subheading.

Degrees of non-persona LLM agency.

There's an image here.

Shoggoths.

On one extreme perspective, the LLM, as depicted by an alien creature called a shoggoth, itself has agency.

The Shoggoth play acts the assistant, the mask, but the Shoggoth is ultimately the one in charge.

This is roughly like a human actor playing a character.

For instance, an actor playing Hamlet could, if he wanted to, distort his portrayal of the character by having Hamlet advocate for the raising of actors' salaries.

However, there is an important disanalogy between actors and Shoggoths.

The Shoggoth is not itself a simulated persona with a human-like psychology.

Its psychology and goals may be alien or inscrutable, as depicted by its bizarre, tentacled form.

On this view, understanding the assistant persona is insufficient for predicting AI assistant behavior because the Shoggoth can in principle override it.

In extreme, out-of-distribution cases, the Shoggoth could even take the mask off fully and start pursuing its alien goals.

There's an image here.

Operating systems.

On an opposing view, the LLM, both before and after post-training, is not too different from a predictive model with no agency of its own.

Pre-trained LLMs are typically viewed this way.

They simply predict probable continuations without having their own agency.

Any agentic outputs are due to the simulated personas, not the underlying LLM.