Eliezer Yudkowsky

For while the strength of the evidences, 90% versus 10%, is just the same in both cases, they do not behave similarly when combined.

How, oh how, will we account for this?

Well, here's half a technical demonstration of how to represent this difference in probability theory.

The rest you can take on my personal authority or look up in the references.

If the probability of H given E1 equals 90% and the probability of H given E2 is 9%, what is the probability of H given E1 and E2?

If learning E1 is true leads us to assign 90% probability to H, and learning E2 is true leads us to assign 9% probability to H, then what probability should we assign to H if we learn both E1 and E2?

This is simply not something you can calculate in probability theory from the information given.

No, the missing information is not the prior probability of H. E1 and E2 may not be independent of each other.

Suppose that H is my sidewalk is slippery, E1 is my sprinkler is running, and E2 is it's night.

The sidewalk is slippery starting from one minute after the sprinkler starts until just after the sprinkler finishes, and the sprinkler runs for 10 minutes.

So if we know the sprinkler is on, the probability is 90% that the sidewalk is slippery.

The sprinkler is on during 10% of the nighttime, so if we know that it's night, the probability of the sidewalk being slippery is 9%.

If we know that it's night and the sprinkler is on, that is, if we know both facts, the probability of the sidewalk being slippery is 90%.

We can represent this in a graphical model as follows.

Here, the author has the word night, followed by an arrow pointing to the right, followed by a sprinkler, followed by a second arrow pointing to the right, followed by slippery.

That's night, arrow to sprinkler, arrow to slippery.

Whether or not it's night causes the sprinkler to be on or off, and whether the sprinkler is on causes the sidewalk to be slippery or unslippery.

The direction of the arrows is meaningful.

If I wrote night arrow to sprinkler arrow from slippery back to sprinkler, that's the word night followed by an arrow pointing to the right followed by sprinkler followed by an arrow pointing to the left followed by slippery,

This would mean that if I didn't know anything about the sprinkler, the probability of nighttime and slipperiness would be independent of each other.