Chris Lattner

It's kind of not super great for deployment, right?

And so I think that we as an industry have been struggling.

And if you look at what deploying a machine learning model today means is that you'll have researchers who are, I mean, wicked smart, of course, but they're wicked smart at deploying

model architecture and data and calculus.

They're wicked smart in various domains.

They don't wanna know anything about the hardware or deployment or C++ or things like this, right?

And so what's happened is you get people who train the model, they throw it over the fence, and then you have people that try to deploy the model.

Well, every time you have a team A does X, they throw it over the fence, and team B does Y, you have a problem because, of course, it never works the first time.

And so you throw it over the fence.

They figure out, okay, it's too slow.

It won't fit.

It doesn't use the right operator.

The tool crashes, whatever the problem is.

Then they have to throw it back over the fence.

And every time you throw a thing over a fence, it takes three weeks of project managers and meetings and things like this.

And so what we've seen today is that getting models in production can take weeks or months.

It's not atypical.

I talk to lots of people, and you talk about VP of software at some internet company trying to deploy a model.

And they're like, why do I need a team of 45 people?

It's so easy to train a model.