Cal Newport

Okay, they're token producers, right?

And they're deterministic in the sense that if you give it the same input, it'll give you the same output.

But they're unpredictable because the output that you actually get from an LLM is effectively a probability distribution over potential next tokens.

And then you probabilistically select a token from that distribution.

This is why if you ask the same prompt multiple times, you might get different answers because those random selections made outside of the LLM might select differently.

So LLMs are unpredictable, right?

That's why if you say, hey, give me a plan for this, like sometimes the plan will make sense and sometimes it doesn't.

And sometimes it'll seem reasonable, but it'll have a weird aside.

Or if you give it a big scenario and say, what do you want to do next?

It might say something you weren't expecting.

And if it gets little cues and it prompts that we're playing like a sci-fi game, it'll be very sci-fi and maybe it'll do like weird things.

LLM textual output is unpredictable.

That doesn't mean that AI systems that query LLMs are unpredictable because all of the control logic and action is done in human written programs.

Right?

So this idea that it's like these are just these black boxes that maybe at some point will spontaneously start improving themselves makes no sense.

The LLM is a static thing that you can prompt and it will give you text back.

That's it.

And a human controlled program does that prompting.

That is not a setup where there is a... Where is the growing intention here?

Where is the, like, we're going to change our own code?