George Hotz

Reduce ops will take a three-dimensional tensor and turn it into a two-dimensional tensor, or a three-dimensional tensor and turn it into a zero-dimensional tensor. Think like a sum or a max are really the common ones there. And then the fourth type is movement ops. And movement ops are different from the other types because they don't actually require computation.

Reduce ops will take a three-dimensional tensor and turn it into a two-dimensional tensor, or a three-dimensional tensor and turn it into a zero-dimensional tensor. Think like a sum or a max are really the common ones there. And then the fourth type is movement ops. And movement ops are different from the other types because they don't actually require computation.

They require different ways to look at memory. So that includes reshapes, permutes, expands, flips. Those are the main ones, probably.

They require different ways to look at memory. So that includes reshapes, permutes, expands, flips. Those are the main ones, probably.

They require different ways to look at memory. So that includes reshapes, permutes, expands, flips. Those are the main ones, probably.

And convolutions. And every convolution you can imagine, dilated convolutions, strided convolutions, transposed convolutions.

And convolutions. And every convolution you can imagine, dilated convolutions, strided convolutions, transposed convolutions.

And convolutions. And every convolution you can imagine, dilated convolutions, strided convolutions, transposed convolutions.

Sure. So if you type in PyTorch A times B plus C, what this is going to do is it's going to first multiply A and B and store that result into memory. And then it is going to add C by reading that result from memory, reading C from memory, and writing that out to memory. There is way more loads and stores to memory than you need there.

Sure. So if you type in PyTorch A times B plus C, what this is going to do is it's going to first multiply A and B and store that result into memory. And then it is going to add C by reading that result from memory, reading C from memory, and writing that out to memory. There is way more loads and stores to memory than you need there.

Sure. So if you type in PyTorch A times B plus C, what this is going to do is it's going to first multiply A and B and store that result into memory. And then it is going to add C by reading that result from memory, reading C from memory, and writing that out to memory. There is way more loads and stores to memory than you need there.

If you don't actually do A times B as soon as you see it, if you wait until the user actually realizes that tensor, until the laziness actually resolves, you confuse that plus C. This is like, it's the same way Haskell works.

If you don't actually do A times B as soon as you see it, if you wait until the user actually realizes that tensor, until the laziness actually resolves, you confuse that plus C. This is like, it's the same way Haskell works.

If you don't actually do A times B as soon as you see it, if you wait until the user actually realizes that tensor, until the laziness actually resolves, you confuse that plus C. This is like, it's the same way Haskell works.

So TinyGrad's front end looks very similar to PyTorch. I probably could make a perfect or pretty close to perfect interop layer if I really wanted to. I think that there's some things that are nicer about TinyGrad syntax than PyTorch, but the front end looks very Torch-like. You can also load in Onyx models. We have more Onyx tests passing than Core ML.

So TinyGrad's front end looks very similar to PyTorch. I probably could make a perfect or pretty close to perfect interop layer if I really wanted to. I think that there's some things that are nicer about TinyGrad syntax than PyTorch, but the front end looks very Torch-like. You can also load in Onyx models. We have more Onyx tests passing than Core ML.

So TinyGrad's front end looks very similar to PyTorch. I probably could make a perfect or pretty close to perfect interop layer if I really wanted to. I think that there's some things that are nicer about TinyGrad syntax than PyTorch, but the front end looks very Torch-like. You can also load in Onyx models. We have more Onyx tests passing than Core ML.

Okay, so... We'll pass Onyx runtime soon.

Okay, so... We'll pass Onyx runtime soon.

Okay, so... We'll pass Onyx runtime soon.