Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Jaden Schaefer

๐Ÿ‘ค Speaker
1542 total appearances

Appearances Over Time

Podcast Appearances

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

I think because they kind of have this design, it's very forward looking.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

I think this matters because model sizes are continuing to grow.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

And I think as companies are increasingly, you know, expecting lower latency, they're expecting kind of this always on AI service rather than a batch style workload, right?

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

Like you're not going to, I think in the olden days or olden days, but like in the past, it was kind of like, hey, we need like an AI model that's going to go and

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

run through and do this massive project, it's going to get this huge batch done for us, and then we're going to be done.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

When you're looking at like how consumers and how the enterprise is using it today, people are pinging this all day, every day, always, it needs to always be on no one wants latency.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

And so it needs to kind of accommodate for that not just like this huge fluctuating, big usage, and then a lull, it's kind of like we're getting this constant steady usage.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

So

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

beyond just like the performance, I think the power efficiency is a really key part of all of this data centers right now, they're already straining against energy constraints, we have even to the top levels of the government talking about, look, you guys need to be building power, power generation, or you know, power creation in some way alongside your data centers, because there just isn't enough.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

And that cost gets passed on to the consumer.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

Like if you live in an area with a

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

And they're all getting subsidized.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

You are paying for it.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

Essentially, your power is going to be more expensive.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

So I think the AI workloads are getting really intense right now.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

But essentially, by designing this chip and creating its own silicone, Microsoft can tune Maya specifically to its data center layouts, which is a really interesting thought.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

Microsoft being, you know, one of the biggest players buying and building these data centers, they could build a chip that's specifically designed for how they structure and run their data centers.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

That's, you know, that's like the cooling systems and it's kind of the software framework and they can do all of this to reduce any sort of wasted power and they can smooth out the deployment at scale.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

So I think that's a really interesting vertical integration.

AI in Business
Microsoft Reveals Maya 200 AI Inference Chip

It's difficult to achieve with any sort of off the shelf GPU alone that you might get from an NVIDIA or anyone else.