Chris Lattner

Auto-tuning is a feature in Mojo.

It's not, so very little of what we're doing is actually research.

Like many of these ideas have existed in other systems and other places, and so what we're doing is we're pulling together good ideas, remixing them, and making them into hopefully a beautiful system, right?

And so auto-tuning, the observation is that

Turns out hardware systems algorithms are really complicated.

Turns out maybe you don't actually want to know how the hardware works, right?

A lot of people don't, right?

And so there are lots of really smart hardware people, I know a lot of them, where they know everything about, okay, the cache size is this and the number of registers is that.

And if you use this length of vector, it's going to be super efficient because it maps directly onto what it can do.

And like,

all this kind of stuff, or the GPU has SMs and it has a warp size of whatever, right?

All the stuff that goes into these things, or the tile size of a TPU is 128, like these factoids, right?

My belief is that most normal people, and I love hardware people also, I'm not trying to offend literally everybody in the internet, but most programmers actually don't want to know this stuff, right?

And so if you come at it from the perspective of how do we allow people to build both more abstracted but also more portable code,

Because, you know, it could be that the vector length changes or the cache size changes, or it could be that the tile size of your matrix changes or the number, you know, an A100 versus an H100 versus a Volta versus a whatever GPU have different characteristics, right?

A lot of the algorithms that you run are actually the same, but the parameters, these magic numbers you have to fill in end up being really fiddly numbers that an expert has to go figure out.

And so what autotuning does, it says, okay, well, guess what?

There's a lot of compute out there.

So instead of having humans go randomly try all the things or do a grid search or go search some complicated multidimensional space, how about we have computers do that?

And so what autotuning does is you can say, hey, here's my algorithm.