Nathan Lambert
speaker
1,814 appearances
3 recordings
2 series
first heard Feb 2025
last heard 1 Feb
Nathan Lambert’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 2 in all, peaking in Feb 2026 with 1.
Appearances
And this drone company just tells the military customers like, hey, just get it from Amazon because I can't actually physically get them, right? Like there's all these things that are happening that point to further and further divergence. I have zero idea. And I would love if we could We could all hold hands and sing Kumbaya, but I have zero idea how that could possibly happen.
It's an objective fact that the world has been the most peaceful it has ever been when there are global hegemons, right? Or regional hegemons, right? In historical context, right? The Mediterranean was the most peaceful ever when the Romans were there, right?
China had very peaceful and warring times, and the peaceful times were when dynasties had a lockhold over not just themselves, but all their tributaries around them, right? And likewise, the most peaceful time in human history has been when the US was the global hegemon, right? The last, you know, decades. Now, we've sort of seen things start to slide, right?
With Russia, Ukraine, with what's going on in the Middle East and, you know, Taiwan risk, all these different things are starting to bubble up, still objectively extremely peaceful. Now, what happens when it's not one global hegemon, but it's two, obviously, and, you know, China will be, you know, competitive or even overtake the US like it's possible, right? And so this change in global hegemony
I don't think it ever happens like super peacefully, right? When empires fall, right, which is a possible trajectory for America, they don't fall gracefully, right? Like they don't just slide out of irrelevance. Usually there's a lot of shaking. And so, you know, what the US is trying to do is maintain its top position. And what China is trying to do is become the top position, right?
And obviously there's butting of heads here in the most simple terms.
And the U.S. 's current task is like, hey, if we control AI, if we're the leader in AI, and AI significantly accelerates progress, then we can maintain the global hegemony position. I hope that works. And as an American, like, you know, kind of like, okay, I guess that's going to lead to peace for us. Now, obviously, other people around the world get affected negatively.
You know, obviously, the Chinese people are not going to be in as advantageous of a position if that happens. But, you know, this is sort of the reality of like what's being done and the actions that are being carried out.
Yeah, so this goes, and I think we'd have to like, we need to dive really deep into the reasoning aspect and what's going on there. But the H20, you know, the US has gone through multiple iterations of the export controls, right? This H800 was at one point allowed back in 23, but then it got canceled. And by then, you know, DeepSeek had already built their cluster of, they claim 2K.
I think they actually have like many more, like something like 10K of those. And now this H20 is the legally allowed chip, right? NVIDIA shipped a million of these last year to China. For context, it was like four or five million GPUs. So the percentage of GPUs that were this China-specific H20 is quite high, roughly 20%, 25%, 20% or so.
And so this H20 has been neutered in one way, but it's actually upgraded in other ways. And you could think of chips along three axes for AI, ignoring software stack and exact architecture, just raw specifications. There's floating point operations, flops. There is memory bandwidth, i.e. in memory capacity, IO, memory. And then there is interconnect, chip-to-chip interconnections.
All three of these are incredibly important for... making AI systems, right? Because AI systems involve a lot of compute. They involve a lot of moving memory around, whether it be to memory or to other chips, right? And so these three vectors, the US initially had two of these vectors controlled and one of them not controlled, which was flops and interconnect bandwidth were initially controlled.
And then they said, no, no, no, no, we're going to remove the interconnect bandwidth and just make it a very simple only flops. But now NVIDIA can now make a chip that has, okay, it's cut down on flops. It's like one third that of the H100 on spec sheet paper performance for flops. In real world, it's closer to like half or maybe even like 60% of it.
But then on the other two vectors, it's just as good for interconnect bandwidth. And then for memory bandwidth and memory capacity, the H20 has more memory bandwidth and and more memory capacity than the H100, right? Now, recently, you know, we at our research, we cut NVIDIA's production for H20 for this year down drastically.
They were going to make another 2 million of those this year, but they just canceled all the orders a couple of weeks ago. In our view, that's because we think that they think they're going to get restricted. Because why would they cancel all these orders for H20? Because they shipped a million of them last year.
They had orders in for a couple million this year and just gone for H20, B20, a successor to H20. And now they're all gone. Now, why would they do this? I think it's very clear. The H20 is actually better for certain tasks. And that certain task is reasoning. right? Reasoning is incredibly different than... When you look at the different regimes of models, right?
Pre-training is all about flops, right? It's all about flops. There's things you do, like mixture of experts that we talked about, to trade off interconnect... Or to trade off other aspects and lower the flops and rely more on interconnect and memory. But at the end of the day, it's flops is everything, right? We talk about models in terms of how many flops they are, right?
So, like, you know, we talk about, oh, GPT-4 is 2E25, right? 2 to the 25th, you know, 25 zeros, right? Flop, right? Floating point operations. For training. For training, right? And we're talking about the restrictions for the 2E24, right?
The US has an executive order that Trump recently unsigned, which was, hey, 1E26, once you hit that number of floating point operations, you must notify the government, And you must share your results with us, right? There's a level of model where the US government must be told, right? And that's 1E26.
And so as we move forward, this is an incredibly important... Flop is the vector that the government has cared about historically, but the other two vectors are arguably just as important, right? And especially when we come to this new paradigm, which the world is only just learning about over the last six months, right? Reasoning.
Showing 1441–1460 of 1,814 · page 73 of 91
← Previous
Next →