Zach Furman

We are no longer surprised when a stove burns us or when water extinguishes a flame because we have learned the underlying process that governs their behavior.

This process of learning only works because the world we inhabit, for all its apparent complexity, is not random.

It is governed by consistent, discoverable rules.

If dropping a glass causes it to shatter on Tuesday, it will do the same on Wednesday.

If one pushes a ball off the top of a hill, it will roll down, at a rate that any high school physics student could predict.

Through our observations, we implicitly reverse-engineer these rules.

This idea, that the physical world is fundamentally predictable and rule-based, has a formal name in computer science.

The Physical Church Turing Thesis

Precisely, it states that any physical process can be simulated to arbitrary accuracy by a Turing machine.

Anything from a star collapsing to a neuron firing can, in principle, be described by an algorithm and simulated on a computer.

From this perspective, one can formalize this notion of building a world model by reverse engineering rules from what we can see.

We can operationalize this as a form of program synthesis.

From observations, attempting to reconstruct some approximation of the true program that generated those observations.

Assuming the physical church during thesis, such a learning algorithm would be universal, able to eventually represent and predict any real-world process.

But this immediately raises a new problem.

For any set of observations, there are infinitely many programs that could have produced them.

How do we choose?

The answer is one of the oldest principles in science.

Occam's razor.

We should prefer the simplest explanation.