Cal Newport

And the LLM will return some code.

Or, okay, we want to create a multi-step plan for investigating why this program is crashing.

I will put that into a prompt.

I will send it to the LLM, say, will you write me a multi-step plan for this goal?

So here's the constraints.

Here's the tools you can use right in this format.

The LLM produces text.

It gets that back.

And now the coding harness, human-written code, completely non-obfuscated, completely deterministic, looks at that plan and says, great, let's break this up into steps and execute them correctly.

one by one.

It's the coding harness that actually has any action, that actually accesses tools.

It's the coding harness that has all the control logic.

So we program that, which means it's eminently controllable.

So let's do a thought experiment here.

Let's say there's a certain tool you don't want your programming tools to use on your computer.

Well, you could do that with 100% certainty.

by just never having your coding harness call that tool.

Just like no matter what instructions it gets, it's just not on the list of tools that it'll call.

This is the thing we often mix up.

LLMs are unpredictable.