Adam Kucharski

But even if the car had been trained to recognize yellow fire trucks, it wouldn't go through the same thought process we do when we see an emergency vehicle.

Self-driving AI views the world as a series of shapes and probabilities.

With sufficient training, it can convert this view into useful actions, but fundamentally, it's not seeing what we're seeing.

This tension between the benefits that computers can bring and the understanding that humans have to relinquish isn't new.

In 1976, two mathematicians named Kenneth Apple and Wolfgang Harkin announced the first ever computer-aided proof.

Their discovery meant that, for the first time in history, mathematicians had to accept a major theorem that they could not verify by hand.

The theorem in question is what's known as the four-color theorem.

In short, this says, if you want to fill in a map with different colors so that no two bordering countries are the same color, you'll only ever need four colors to do this.

The mathematicians had found that there were too many map configurations to construe by hand.

even if they simplify things by looking for symmetries.

So they used a computer to get over the finish line.

Not everyone believed the proof initially.

Maybe the computer had made an error somewhere.

Suddenly, mathematicians no longer had total intellectual control.

They had to trust a machine.

But then something curious happened.

While older researchers had been skeptical, younger mathematicians took the opposite view.

why would they trust hundreds of pages of handwritten and hand-checked calculations?

Surely a computer would be more accurate.

Whether we're talking about anesthesia, self-driving cars, or mathematical proofs, perhaps we don't need to fully understand something as long as the accuracy is high enough for what we need.