Ihor Kendiukhov

I think we failed to notice.

End quote.

The argument, unpacked, goes like this.

The VNM framework, and every axiomatization of utility that Garabrant is aware of, implicitly assumes updating.

When you observe something, say, a coin comes up heads, you condition on that observation and from that point forward you only care about worlds consistent with it.

The worlds where the coin came up tails are discarded from your deliberation.

This is Bayesian updating applied to preferences, not just beliefs, and it is so deeply embedded in the framework that it is usually invisible.

But the less wrong Amiri decision theory research program discovered, through work on updateless decision theory and its successors, that updating is not a requirement of rationality.

An updateless agent does not narrow its caring when it makes an observation.

Now here is the connection which is the reason I am presenting Garabrant's comment at length.

The updating step that Garabrant identifies as the hidden assumption in utility theory is, formally, the same thing as the branch-by-branch evaluation that the independence axiom encodes.

When you update on the coin came up heads, you evaluate your remaining options conditional on this observation, ignoring the tails branch.

Independence says this conditional evaluation should be the same regardless of what was on the tails branch, precisely because you are supposed to discard the tails branch after updating.

An updateless agent, by contrast, evaluates the entire policy, covering both heads and tails, as a single object, and the value of the head's branch action depends on what the tail's branch action is, because both are part of the same globally optimized policy.

This is structurally parallel to the e-critique.

The time-average reasoner evaluates the entire trajectory, all branches, the full compounding structure, as a unified object, rather than decomposing it into independent branches and evaluating each one after updating on which branch was realized.

The EE agent is, in Garabrant's terminology, updateless with respect to the temporal unfolding of their wealth process.

Two completely independent lines of thought, one coming from physics and the mathematics of stochastic processes, the other coming from the philosophical and logical analysis of decision theory within the rationalist community, converge on the same structural conclusion.

The independence axiom encodes a branch-by-branch, post-update evaluation that is not required by rationality, and the most reflectively coherent agents are those who evaluate holistically rather than branch-by-branch.

Subheading Academians of VNM Expected Utility Theory Uses, Abuses, and Interpretation, 2010 Academians post is one of the more thoughtful and careful treatments of VNM utility on less wrong.