Zach Furman

not monolithic black-box computations, but something more like circuits.

This is reminiscent of the picture we started with.

Solomonoff induction frames learning as a search for simple programs that explain data.

It is a theoretical ideal, provably optimal in a certain sense, but hopelessly intractable.

The connection between Solomonoff and deep learning has mostly been viewed as purely conceptual.

A nice way to think about what learning should do with no implications for what neural networks actually do.

But the evidence from mechanistic interpretability suggests a different possibility.

What if deep learning is doing something functionally similar to program synthesis?

Not through the same mechanism, gradient descent on continuous parameters is nothing like enumerative search over discrete programs.

But perhaps targeting the same kind of object.

Mechanistic solutions, built from parts, that capture structure in the data generating process.

To be clear, this is a hypothesis.

The evidence shows that neural networks can learn compositional solutions, and that such solutions have appeared alongside generalization in specific, interpretable cases.

It doesn't show that this is what's always happening, or that there's a consistent bias toward simplicity, or that we understand why gradient descent would find such solutions efficiently.

but if the hypothesis is right, it would reframe what deep learning is doing.

The success of neural networks would not be a mystery to be accepted, but an instance of something we already understand in principle.

The power of searching for compact, mechanistic models to explain your observations.

The puzzle would shift from why does deep learning work at all to how does gradient descent implement this search so efficiently?

That second question is hard.

Solomonoff induction is intractable precisely because the space of programs is vast and discrete.