Terence Tao

A lot of it's the test of time.

So many great ideas didn't actually get a great reception at the time that they were first proposed.

It was only after some other scientists proposed.

realized that they could take it further and apply them to their own.

Deep learning itself was actually a niche area of AI for a long time.

The idea of getting answers entirely through training on data and not through first principles reasoning was very controversial.

And it took a long time before it actually started bearing fruit.

you know, you mentioned the bit, you know, I mean, there are other proposals for computer architectures than the zero one that is universal today.

I think there were trits, you know, zero one, three valued logic and, you know, in an alternate universe, maybe a different paradigm would have showed up.

People have argued that the transformer, for example, is the foundation of all modern large language models.

And it was the first deep learning architecture that really was sophisticated enough to capture language.

But it didn't have to be that way.

There could have been some other architecture that...

was the first to do it.

And once that was adopted, it would become the standard.

So I think one reason why it's hard to assess whether a given idea is going to be fruitful is that it depends on the future.

It depends also on the culture and society, like which ones get adopted, which ones don't.

The base 10 numeral system in mathematics is extremely useful, much better than the Roman numeral system, for instance.

But again, there's nothing special about 10.

It's a system that is useful for us because everyone else uses it.