Kevin Roose
๐ค SpeakerAppearances Over Time
Podcast Appearances
It would be like if you just bought like a very high-end sports car, like a Lamborghini, and you had been driving it around and were so proud of how fast it could accelerate and how well it handled. And then like some random guy shows up with like a soapbox car made of balsa wood, and it can go just as fast as your car. You'd be like, what the heck?
Why did I just spend all this money on this Lamborghini?
Why did I just spend all this money on this Lamborghini?
Yes. And then, of course, there's the geopolitical freakout because DeepSeek is a Chinese AI company. And there has been this race happening between primarily the U.S. and China for years about AI and AI supremacy. Who was going to be able to build the most powerful AI models before the other one? And that is a very important question for things like assessing the future of military conflict.
Yes. And then, of course, there's the geopolitical freakout because DeepSeek is a Chinese AI company. And there has been this race happening between primarily the U.S. and China for years about AI and AI supremacy. Who was going to be able to build the most powerful AI models before the other one? And that is a very important question for things like assessing the future of military conflict.
If one country's AI is way better than another country's AI, they might have an advantage. In fact, the U.S. has banned the export of the most powerful AI chips to China for exactly this reason, to try to... sort of hobble the Chinese AI companies to keep them from catching up when it comes to building the bleeding edge models that could become very important.
If one country's AI is way better than another country's AI, they might have an advantage. In fact, the U.S. has banned the export of the most powerful AI chips to China for exactly this reason, to try to... sort of hobble the Chinese AI companies to keep them from catching up when it comes to building the bleeding edge models that could become very important.
So instead, DeepSeq had to kind of make do with these like Kirkland signature chips that are, you know, pretty good, but they're not the best. And so that combined with the amount of money spent really made people say, how do they pull this thing off?
So instead, DeepSeq had to kind of make do with these like Kirkland signature chips that are, you know, pretty good, but they're not the best. And so that combined with the amount of money spent really made people say, how do they pull this thing off?
Yeah, so there are a lot of people who are skeptical of what DeepSeek has claimed. In particular, the cost of the model, $5.5 million might not be the real figure. It doesn't include all of the research and the engineer salaries and things that went into that, so that the real cost is probably significantly higher than that.
Yeah, so there are a lot of people who are skeptical of what DeepSeek has claimed. In particular, the cost of the model, $5.5 million might not be the real figure. It doesn't include all of the research and the engineer salaries and things that went into that, so that the real cost is probably significantly higher than that.
But there are questions about, you know, did they smuggle in very powerful chips that would have actually allowed them to build a model this good? Hmm. You know, is there something going on? Is the Chinese government funneling money to them and not telling us about it? So there are lots of theories.
But there are questions about, you know, did they smuggle in very powerful chips that would have actually allowed them to build a model this good? Hmm. You know, is there something going on? Is the Chinese government funneling money to them and not telling us about it? So there are lots of theories.
But then as time wears on and people who are experts in this stuff start digging through the details, they're coming to the conclusion that, well, yeah, Maybe the cost is a little higher than DeepSea claims. Maybe they have a few more chips than they're telling us about. But in general, this seems like they actually just did build a really good model using some very clever engineering techniques.
But then as time wears on and people who are experts in this stuff start digging through the details, they're coming to the conclusion that, well, yeah, Maybe the cost is a little higher than DeepSea claims. Maybe they have a few more chips than they're telling us about. But in general, this seems like they actually just did build a really good model using some very clever engineering techniques.
So because DeepSeq did not have access, we don't think, to the most powerful chips that American companies are using... they had to kind of get clever about becoming more efficient with their model. I won't bore you with the technical details.
So because DeepSeq did not have access, we don't think, to the most powerful chips that American companies are using... they had to kind of get clever about becoming more efficient with their model. I won't bore you with the technical details.
It includes terms like mixture of experts, architecture, but basically they were able to use some clever tricks to squeeze the most power out of the chips that they did have.
It includes terms like mixture of experts, architecture, but basically they were able to use some clever tricks to squeeze the most power out of the chips that they did have.
Yeah, I mean, there's this saying in the tech industry that constraints inspire creativity. And that is definitely true here. DeepSeq did not have access to the best American AI chips. They did not have the largest budget or the most sophisticated team, but they were really scrappy and smart. They had a lot of really good young engineers and they were able to pull this off.