John Siracusa

uh and better uh and with a slightly different approach and that was an optimization they made so yeah like i mean this is this is not as severe as that because what they've done is basically just a really good job of programming the hardware they had anyway we should we should go to this next item because it goes into more detail about the particular innovations they they made

uh and better uh and with a slightly different approach and that was an optimization they made so yeah like i mean this is this is not as severe as that because what they've done is basically just a really good job of programming the hardware they had anyway we should we should go to this next item because it goes into more detail about the particular innovations they they made

Yeah, so these innovations they had, like, again, some of the innovations are things that OpenAI was already doing with GPT, and they're doing as well. And then the other thing is, you know, if you look at the paper, it's like, oh, well, you have a bunch of data, and it's very expensive. What if you compressed it? I mean, it's not rocket science, but that's how innovation goes here. Like, we...

Yeah, so these innovations they had, like, again, some of the innovations are things that OpenAI was already doing with GPT, and they're doing as well. And then the other thing is, you know, if you look at the paper, it's like, oh, well, you have a bunch of data, and it's very expensive. What if you compressed it? I mean, it's not rocket science, but that's how innovation goes here. Like, we...

You know, let's let's take the approach that they did with GPT-4 and do that same thing to reduce our footprint. And let's reduce it further by compressing this thing that used to take up a lot of memory. And on the cost front, these are by the way, these are papers are not they're not about the R1 model that we're talking about. These are precursors to that.

You know, let's let's take the approach that they did with GPT-4 and do that same thing to reduce our footprint. And let's reduce it further by compressing this thing that used to take up a lot of memory. And on the cost front, these are by the way, these are papers are not they're not about the R1 model that we're talking about. These are precursors to that.

So if you'd been paying attention to this stuff a month or two ago when they put the stuff out, you could have seen that like what was coming. But on the cost thing, they said it would only cost us six million dollars. When they originally said that, a lot of people just didn't believe them. Because they said, well, they're lying. It's a Chinese company.

So if you'd been paying attention to this stuff a month or two ago when they put the stuff out, you could have seen that like what was coming. But on the cost thing, they said it would only cost us six million dollars. When they originally said that, a lot of people just didn't believe them. Because they said, well, they're lying. It's a Chinese company.

They're lying about how much money it costs. There's no way they could have spent $6 million to do something that costs hundreds of millions of dollars when OpenAI did it and have equal performance. It's obviously not true.

They're lying about how much money it costs. There's no way they could have spent $6 million to do something that costs hundreds of millions of dollars when OpenAI did it and have equal performance. It's obviously not true.

Yeah, exactly. But in the paper, they do say that those costs are not it's not the all in cost. That's just the cost of their final run. But that's, you know, and maybe the open AI number is like all the research needed to get to that point. So maybe the number is not as low as they say it is.

Yeah, exactly. But in the paper, they do say that those costs are not it's not the all in cost. That's just the cost of their final run. But that's, you know, and maybe the open AI number is like all the research needed to get to that point. So maybe the number is not as low as they say it is.

But you can see in Ben's paper, he does some back of the envelope math to say, given the technology that they've described in their own public research papers, you can do the math and say.

But you can see in Ben's paper, he does some back of the envelope math to say, given the technology that they've described in their own public research papers, you can do the math and say.

yeah they're if the number is not exact it's in the ballpark you can see how they would arrive at that because they're they're doing less stuff like that's the innovation do do less math in computers like use less memory do less computations and the magic is that when you do less work and spend less money you can somehow get a result that is comparable to open ai right it's

yeah they're if the number is not exact it's in the ballpark you can see how they would arrive at that because they're they're doing less stuff like that's the innovation do do less math in computers like use less memory do less computations and the magic is that when you do less work and spend less money you can somehow get a result that is comparable to open ai right it's

So I think that mostly holds out. So one of the other theories, speaking of the people who thought the iPhone was faked or whatever, the theory was like, they must have gotten the good GPUs. They couldn't be using NVIDIA H800 or whatever things. They must have gotten the good ones secretly. No.

So I think that mostly holds out. So one of the other theories, speaking of the people who thought the iPhone was faked or whatever, the theory was like, they must have gotten the good GPUs. They couldn't be using NVIDIA H800 or whatever things. They must have gotten the good ones secretly. No.

they just figured out they did the, like the equivalent of like writing an assembly code, like the low level version of like, uh, extracting every ounce of juice from the crappy GPUs that they do have that. I mean, just straight up, like just brute force.

they just figured out they did the, like the equivalent of like writing an assembly code, like the low level version of like, uh, extracting every ounce of juice from the crappy GPUs that they do have that. I mean, just straight up, like just brute force.