Nathan Lambert

Whereas if they just launched one, it was like 30%.

Whereas if they just launched one, it was like 30%.

Whereas if they just launched one, it was like 30%.

So the fantastic thing is, and it's in the thing that I pulled up earlier, but the cost for GPT-3 has plummeted, if you scroll up just a few images, I think.

So the fantastic thing is, and it's in the thing that I pulled up earlier, but the cost for GPT-3 has plummeted, if you scroll up just a few images, I think.

So the fantastic thing is, and it's in the thing that I pulled up earlier, but the cost for GPT-3 has plummeted, if you scroll up just a few images, I think.

important thing about like hey is cost a limiting factor here right like my my view is that like we'll have like really awesome intelligence before we have like agi before we have it permeate throughout the economy um and this is sort of why that reason is right gpt3 was trained in what 2020 2021 um and the cost for running inference on it was 60 70 per million tokens right um which is the cost per intelligence was ridiculous

important thing about like hey is cost a limiting factor here right like my my view is that like we'll have like really awesome intelligence before we have like agi before we have it permeate throughout the economy um and this is sort of why that reason is right gpt3 was trained in what 2020 2021 um and the cost for running inference on it was 60 70 per million tokens right um which is the cost per intelligence was ridiculous

important thing about like hey is cost a limiting factor here right like my my view is that like we'll have like really awesome intelligence before we have like agi before we have it permeate throughout the economy um and this is sort of why that reason is right gpt3 was trained in what 2020 2021 um and the cost for running inference on it was 60 70 per million tokens right um which is the cost per intelligence was ridiculous

Now, as we scaled forward two years, we've had a 1200x reduction in cost to achieve the same level of intelligence as GPT-3.

Now, as we scaled forward two years, we've had a 1200x reduction in cost to achieve the same level of intelligence as GPT-3.

Now, as we scaled forward two years, we've had a 1200x reduction in cost to achieve the same level of intelligence as GPT-3.

It's like 5 cents or something like that now, right? Which is versus $60, 1200x. That's not the exact numbers, but it's 1200x. I remember that number. Is... is the humongous cost per intelligence, right? Now, the freak out over DeepSeq is, oh my God, they made it so cheap.

It's like 5 cents or something like that now, right? Which is versus $60, 1200x. That's not the exact numbers, but it's 1200x. I remember that number. Is... is the humongous cost per intelligence, right? Now, the freak out over DeepSeq is, oh my God, they made it so cheap.

It's like 5 cents or something like that now, right? Which is versus $60, 1200x. That's not the exact numbers, but it's 1200x. I remember that number. Is... is the humongous cost per intelligence, right? Now, the freak out over DeepSeq is, oh my God, they made it so cheap.

It's like, actually, if you look at this trend line, they're not below the trend line, first of all, and at least for GPT-3, right? They are the first to hit it, right? Which is a big deal. But they're not below the trend line as far as GPT-3. Now we have GPT-4. What's going to happen with these reasoning capabilities, right? It's a mix of architectural innovations.

It's like, actually, if you look at this trend line, they're not below the trend line, first of all, and at least for GPT-3, right? They are the first to hit it, right? Which is a big deal. But they're not below the trend line as far as GPT-3. Now we have GPT-4. What's going to happen with these reasoning capabilities, right? It's a mix of architectural innovations.

It's like, actually, if you look at this trend line, they're not below the trend line, first of all, and at least for GPT-3, right? They are the first to hit it, right? Which is a big deal. But they're not below the trend line as far as GPT-3. Now we have GPT-4. What's going to happen with these reasoning capabilities, right? It's a mix of architectural innovations.

It's a mix of better data, and it's going to be better training techniques, and all of these better inference systems uh, better hardware, right. Uh, going from, you know, each generation of GPU to new generations or a six, everything is going to take this cost curve down and down and down and down.

It's a mix of better data, and it's going to be better training techniques, and all of these better inference systems uh, better hardware, right. Uh, going from, you know, each generation of GPU to new generations or a six, everything is going to take this cost curve down and down and down and down.