Chris Lattner

I have generally a graph of compute.

I want to do a partitioning.

I want to look at the communication, the bisection bandwidth and like the overhead and the sending of all these different things and build a model for this and then decide, okay, it's an optimization problem.

Where do I want to place this compute?

Absolutely.

So, I mean, in my opinion, this is an opinion, this is not, not everybody would agree with this, but in my opinion, the world benefits from simple and predictable systems at the bottom that you can control.

But then once you have a predictable execution layer, you can build lots of different policies on top of it.

And so one policy can be that the human programmer says, do that here, do that here, do that here, do that here, and fully manually controls everything, and the system should just do it.

Then you quickly get in the mode of, I don't want to have to tell it to do it.

And so the next logical step that people typically take is they write some terrible heuristic.

If it's a matrix multiplication, do it over there.

Or if it's floating point, do it on the GPU.

If it's integer, do it on the CPU.

Something like that, right?

And then you then get into this mode of people care more and more and more, and you say, okay, well, let's actually make the heuristic better.

Let's get into auto-tuning.

Let's actually do a search of the space to decide, well, what is actually better, right?

Well, then you get into this problem where you realize this is not a small space.

This is a many-dimensional...

hyperdimensional space that you cannot exhaustively search.