Nathan Lambert

But it just had the interconnect bandwidth cut. DeepSeq knew how to utilize this. Hey, even though we're cut back on the interconnect, we can do all this fancy stuff to figure out how to use the GPU fully anyways. And so that was back in October 2022. But later in 2023, end of 2023, implemented in 2024, the US government banned the H800, right? Yeah.

And so, by the way, this H800 cluster, these 2,000 GPUs, was not even purchased in 2024, right? It was purchased in late 2023. And they're just getting the model out now, right, because it takes a lot of research, et cetera. H800 was banned, and now there's a new chip called the H20. The H20 is cut back on only flops, but the interconnect bandwidth is the same.

And so, by the way, this H800 cluster, these 2,000 GPUs, was not even purchased in 2024, right? It was purchased in late 2023. And they're just getting the model out now, right, because it takes a lot of research, et cetera. H800 was banned, and now there's a new chip called the H20. The H20 is cut back on only flops, but the interconnect bandwidth is the same.

And so, by the way, this H800 cluster, these 2,000 GPUs, was not even purchased in 2024, right? It was purchased in late 2023. And they're just getting the model out now, right, because it takes a lot of research, et cetera. H800 was banned, and now there's a new chip called the H20. The H20 is cut back on only flops, but the interconnect bandwidth is the same.

And in fact, in some ways, it's better than the H100 because it has better memory bandwidth and memory capacity. So there are, you know, NVIDIA is working within the constraints of what the government says and then builds the best possible GPU for China.

And in fact, in some ways, it's better than the H100 because it has better memory bandwidth and memory capacity. So there are, you know, NVIDIA is working within the constraints of what the government says and then builds the best possible GPU for China.

And in fact, in some ways, it's better than the H100 because it has better memory bandwidth and memory capacity. So there are, you know, NVIDIA is working within the constraints of what the government says and then builds the best possible GPU for China.

To some extent, training a model does effectively nothing, right? Yeah. The thing that Dario is sort of speaking to is the implementation of that model once trained to then create huge economic growth, huge increases in military capabilities, huge increases in productivity of people, betterment of lives, whatever you want to direct super powerful AI towards, you can't.

To some extent, training a model does effectively nothing, right? Yeah. The thing that Dario is sort of speaking to is the implementation of that model once trained to then create huge economic growth, huge increases in military capabilities, huge increases in productivity of people, betterment of lives, whatever you want to direct super powerful AI towards, you can't.

To some extent, training a model does effectively nothing, right? Yeah. The thing that Dario is sort of speaking to is the implementation of that model once trained to then create huge economic growth, huge increases in military capabilities, huge increases in productivity of people, betterment of lives, whatever you want to direct super powerful AI towards, you can't.

But that requires significant amounts of compute, right? And so the US government has effectively said, And forever, right? Training will always be a portion of the total compute. We mentioned Meta's 400,000 GPUs, only 16,000 made Lama, right?

But that requires significant amounts of compute, right? And so the US government has effectively said, And forever, right? Training will always be a portion of the total compute. We mentioned Meta's 400,000 GPUs, only 16,000 made Lama, right?

But that requires significant amounts of compute, right? And so the US government has effectively said, And forever, right? Training will always be a portion of the total compute. We mentioned Meta's 400,000 GPUs, only 16,000 made Lama, right?

So the percentage that Meta's dedicating to inference, now this might be for recommendation systems that are trying to hack our mind into spending more time and watching more ads, or if it's for a super powerful AI that's doing productive things, doesn't matter about the exact use that our economic system decides, it's that that can be delivered in whatever way we want. Whereas with China, right.

So the percentage that Meta's dedicating to inference, now this might be for recommendation systems that are trying to hack our mind into spending more time and watching more ads, or if it's for a super powerful AI that's doing productive things, doesn't matter about the exact use that our economic system decides, it's that that can be delivered in whatever way we want. Whereas with China, right.

So the percentage that Meta's dedicating to inference, now this might be for recommendation systems that are trying to hack our mind into spending more time and watching more ads, or if it's for a super powerful AI that's doing productive things, doesn't matter about the exact use that our economic system decides, it's that that can be delivered in whatever way we want. Whereas with China, right.

You know, you're, you know, expert restrictions. Great. You're never going to be able to cut everything off. Right. And that's, that's like, I think that's quite well understood by the U S government is that you can't cut everything off. You know, they'll make their own chips and they're trying to make their own chips. They'll be worse than ours.

You know, you're, you know, expert restrictions. Great. You're never going to be able to cut everything off. Right. And that's, that's like, I think that's quite well understood by the U S government is that you can't cut everything off. You know, they'll make their own chips and they're trying to make their own chips. They'll be worse than ours.

You know, you're, you know, expert restrictions. Great. You're never going to be able to cut everything off. Right. And that's, that's like, I think that's quite well understood by the U S government is that you can't cut everything off. You know, they'll make their own chips and they're trying to make their own chips. They'll be worse than ours.

But you know, this is the whole point is to just keep a gap. Right. And therefore, at some point as the AI, you know, in a world where two, three percent economic growth, this is really dumb, by the way, right, to cut off, you know, high tech and not make money off of it.