Jensen Huang

Hello.

Hello.

Hi, everybody. It's great to join you at Acquired Live. Wow, this is really something. I still remember when I met Dave and Ben. They interviewed me right here on this stage at NVIDIA's headquarters. And now this podcast has attracted an incredible audience. And so I'm really proud of them.

Hi, everybody. It's great to join you at Acquired Live. Wow, this is really something. I still remember when I met Dave and Ben. They interviewed me right here on this stage at NVIDIA's headquarters. And now this podcast has attracted an incredible audience. And so I'm really proud of them.

And one of the questions that they asked me was, you know, if I knew what I know now, I think, or something like that, you know, would I start NVIDIA all over again? And I said, absolutely not. Of course, it was taken out of context because I was asked about that several times after that. And of course I would start the company if I knew it would turn out this way.

And one of the questions that they asked me was, you know, if I knew what I know now, I think, or something like that, you know, would I start NVIDIA all over again? And I said, absolutely not. Of course, it was taken out of context because I was asked about that several times after that. And of course I would start the company if I knew it would turn out this way.

The reason why I said what I said was it has everything to do with being an entrepreneur. Building a company is insanely hard. The number of things that you have to know, the amazing people that you have to surround yourself with, the adversaries and all the smart things that they're going to do, and the

The reason why I said what I said was it has everything to do with being an entrepreneur. Building a company is insanely hard. The number of things that you have to know, the amazing people that you have to surround yourself with, the adversaries and all the smart things that they're going to do, and the

and the adversities that you're going to be confronted with over time, the mountain of it in the course of 31 years, if I were to take all of that, all the challenges and all the hardship and all the pain and suffering of the last 32 years, and I would have compressed it into the brain of a 29-year-old, there is no way that that person would have started the company.

and the adversities that you're going to be confronted with over time, the mountain of it in the course of 31 years, if I were to take all of that, all the challenges and all the hardship and all the pain and suffering of the last 32 years, and I would have compressed it into the brain of a 29-year-old, there is no way that that person would have started the company.

And my point there is the super point of entrepreneurs, which is your superpowers are partly your ignorance, that you don't know how hard it is. And so, I mean, that's what I meant.

And my point there is the super point of entrepreneurs, which is your superpowers are partly your ignorance, that you don't know how hard it is. And so, I mean, that's what I meant.

When you realize this, you realize, hang on a second, a universal function approximator, a machine learning system, something that learns from examples, could have tremendous opportunities because just the number of applications is quite enormous. And everything from, obviously, we just talked about commerce all the way to science.

When you realize this, you realize, hang on a second, a universal function approximator, a machine learning system, something that learns from examples, could have tremendous opportunities because just the number of applications is quite enormous. And everything from, obviously, we just talked about commerce all the way to science.

And so you realize that maybe this could affect a very large part of the world's industries. Almost every piece of software in the world would eventually be programmed this way. And if that's the case, then how you build a computer and how you build a chip, in fact, can be completely changed.

And so you realize that maybe this could affect a very large part of the world's industries. Almost every piece of software in the world would eventually be programmed this way. And if that's the case, then how you build a computer and how you build a chip, in fact, can be completely changed.

And realizing that, the rest of it just comes with, you know, do you have the courage to put your chips behind it?

And realizing that, the rest of it just comes with, you know, do you have the courage to put your chips behind it?

The first thought was, of course, reasoning about how we should change our computing stack. The second thought is where can we find earliest possibilities of use? If we were to go build this computer, what would people use it to do?

The first thought was, of course, reasoning about how we should change our computing stack. The second thought is where can we find earliest possibilities of use? If we were to go build this computer, what would people use it to do?

And we were fortunate that working with the world's universities and researchers was innate in our company because we were already working on CUDA, and CUDA's early adopters were researchers because we democratized supercomputing. You know, CUDA is not just used, as you know, for AI. CUDA is used for almost all fields of science.

And we were fortunate that working with the world's universities and researchers was innate in our company because we were already working on CUDA, and CUDA's early adopters were researchers because we democratized supercomputing. You know, CUDA is not just used, as you know, for AI. CUDA is used for almost all fields of science.

Everything from molecular dynamics to imaging, CT reconstruction, to seismic processing, to weather simulations, quantum chemistry, the list goes on, right? And so the number of applications of CUDA in research was very high. And so when the time came and we realized that deep learning could be really interesting, it was natural for us to go back to the researchers

Everything from molecular dynamics to imaging, CT reconstruction, to seismic processing, to weather simulations, quantum chemistry, the list goes on, right? And so the number of applications of CUDA in research was very high. And so when the time came and we realized that deep learning could be really interesting, it was natural for us to go back to the researchers

and find every single AI researcher on the planet and say, how can we help you advance your work? And that included Yann LeCun and Andrew Eng and Jeff Hinton. And that's how I met all these people. And I used to go to all the AI conferences and that's where, you know, I met Ilya Suscomber there for the first time. Yeah.

and find every single AI researcher on the planet and say, how can we help you advance your work? And that included Yann LeCun and Andrew Eng and Jeff Hinton. And that's how I met all these people. And I used to go to all the AI conferences and that's where, you know, I met Ilya Suscomber there for the first time. Yeah.

And so it was really about, at that point, what are the systems that we can build and the software stacks we can build to help you be more successful to advance the research? Because at the time, it looked like a toy. But we had confidence that even GAN, the first time I met Goodfellow, the GAN was like 32 by 32. And it was just a blurry image of a cat, but how far can it go?

And so it was really about, at that point, what are the systems that we can build and the software stacks we can build to help you be more successful to advance the research? Because at the time, it looked like a toy. But we had confidence that even GAN, the first time I met Goodfellow, the GAN was like 32 by 32. And it was just a blurry image of a cat, but how far can it go?

And so we believed in it. We believed that one, you could scale deep learning because obviously it's trained layer by layer and you could make the data sets larger and you could make the models larger. And we believe that if you made that larger and larger, it would get better and better, kind of sensible.

And so we believed in it. We believed that one, you could scale deep learning because obviously it's trained layer by layer and you could make the data sets larger and you could make the models larger. And we believe that if you made that larger and larger, it would get better and better, kind of sensible.

I think the discussions and the engagements with the researchers was the exact positive feedback system that we needed. I would go back to research. That's where it all happened.

I think the discussions and the engagements with the researchers was the exact positive feedback system that we needed. I would go back to research. That's where it all happened.

I wasn't involved in the founding of it, but I knew a lot of the people there. And Elon, of course, I knew. And Peter Beale was there. And Ilya was there. And we have some great employees today that were there in the beginning.

I wasn't involved in the founding of it, but I knew a lot of the people there. And Elon, of course, I knew. And Peter Beale was there. And Ilya was there. And we have some great employees today that were there in the beginning.

I knew that they needed this amazing computer that we were building, and we're building the first version of the DGX, which today when you see a hopper, it's 70 pounds, 35,000 parts, 10,000 amps. But DGX, the first version that we built, was used internally, and I delivered the first one to OpenAI, and that was a fun day.

I knew that they needed this amazing computer that we were building, and we're building the first version of the DGX, which today when you see a hopper, it's 70 pounds, 35,000 parts, 10,000 amps. But DGX, the first version that we built, was used internally, and I delivered the first one to OpenAI, and that was a fun day.

Most of our success was aligned around, in the beginning, just about helping the researchers get to the next level. I knew it wasn't very useful in its current state, but I also believed that in a few clicks, it could be really remarkable. And that belief system came from the interactions with all these amazing researchers, and it came from just seeing the incremental progress.

Most of our success was aligned around, in the beginning, just about helping the researchers get to the next level. I knew it wasn't very useful in its current state, but I also believed that in a few clicks, it could be really remarkable. And that belief system came from the interactions with all these amazing researchers, and it came from just seeing the incremental progress.

At first, the papers were coming out every three months, and then papers today are coming out every day, right? So you could just monitor the archive papers, and I took an interest in learning about the progress of deep learning, and to the best of my ability, read these papers, and you could just see the progress happening, you know, in real time, exponentially in real time.

At first, the papers were coming out every three months, and then papers today are coming out every day, right? So you could just monitor the archive papers, and I took an interest in learning about the progress of deep learning, and to the best of my ability, read these papers, and you could just see the progress happening, you know, in real time, exponentially in real time.

Well, my first feeling about the language model was how clever it was to just mask out words and make it predict the next word. It's self-supervised learning at its best. We have all this text, you know, I know what the answer is. I'll just make you guess it. And so my first impression of BERT was really how clever it was. And now the question is, how can you scale that?

Well, my first feeling about the language model was how clever it was to just mask out words and make it predict the next word. It's self-supervised learning at its best. We have all this text, you know, I know what the answer is. I'll just make you guess it. And so my first impression of BERT was really how clever it was. And now the question is, how can you scale that?

You know, the first observation on almost everything is interesting, and then try to understand intuitively why it works. And then the next step, of course, is from first principles, how would you extrapolate that? And so obviously we knew that BERT was going to be a lot larger. Now, one of the things about these language models is it's encoding information, isn't that right?

You know, the first observation on almost everything is interesting, and then try to understand intuitively why it works. And then the next step, of course, is from first principles, how would you extrapolate that? And so obviously we knew that BERT was going to be a lot larger. Now, one of the things about these language models is it's encoding information, isn't that right?

It's compressing information. And so within the world's languages and text, there's a fair amount of reasoning that's encoded in it. We describe a lot of reasoning things, and so if you were to say that few-step reasoning is somehow learnable from just reading things, I wouldn't be surprised. For a lot of us, we get our common sense and we get our reasoning ability by reading.

It's compressing information. And so within the world's languages and text, there's a fair amount of reasoning that's encoded in it. We describe a lot of reasoning things, and so if you were to say that few-step reasoning is somehow learnable from just reading things, I wouldn't be surprised. For a lot of us, we get our common sense and we get our reasoning ability by reading.

And so why wouldn't a machine learning model also learn some of the reasoning capabilities from that? And from reasoning capabilities, you could have emergent capabilities, right? Emergent abilities are consistent with intuitively from reasoning. And so some of it could be predictable, but still, it's still amazing. The fact that it's sensible doesn't make it any less amazing. Right.

And so why wouldn't a machine learning model also learn some of the reasoning capabilities from that? And from reasoning capabilities, you could have emergent capabilities, right? Emergent abilities are consistent with intuitively from reasoning. And so some of it could be predictable, but still, it's still amazing. The fact that it's sensible doesn't make it any less amazing. Right.

I could visualize literally the entire computer and all the modules in a self-driving car. And the fact that it's still keeping lanes makes me insanely happy. And so...

I could visualize literally the entire computer and all the modules in a self-driving car. And the fact that it's still keeping lanes makes me insanely happy. And so...

Yeah, yeah. Exactly. Yeah, yeah. When you put it all together, it's still a miracle. Yeah.

Yeah, yeah. Exactly. Yeah, yeah. When you put it all together, it's still a miracle. Yeah.

I don't think it's something special with NVIDIA. I think that we had the courage to build a system like this. NVIDIA is not built like a military. It's not built like the armed forces where you have generals and colonels. We're not set up like that. We're not set up in a command and control and information distribution system from the top down.

I don't think it's something special with NVIDIA. I think that we had the courage to build a system like this. NVIDIA is not built like a military. It's not built like the armed forces where you have generals and colonels. We're not set up like that. We're not set up in a command and control and information distribution system from the top down.

we're really built much more like a computing stack. And a computing stack, the lowest layer is our architecture, and then there's our chip, and then there's our software, and on top of it, there are all these different modules, and each one of these layers of modules are people.

we're really built much more like a computing stack. And a computing stack, the lowest layer is our architecture, and then there's our chip, and then there's our software, and on top of it, there are all these different modules, and each one of these layers of modules are people.

And so the architecture of the company, to me, is a computer with a computing stack with people managing different parts of the system. and who reports to whom, your title is not related to anywhere you are in the stack. It just happens to be who is the best at running that module on that function, on that layer, it is in charge, and that person is the pilot in command.

And so the architecture of the company, to me, is a computer with a computing stack with people managing different parts of the system. and who reports to whom, your title is not related to anywhere you are in the stack. It just happens to be who is the best at running that module on that function, on that layer, it is in charge, and that person is the pilot in command.

So that's one characteristic.

So that's one characteristic.

Yeah, pretty much. The reason for that is because your organization should be the architecture of the machinery of building the product. That's what a company is. And yet everybody's company look exactly the same, but they all build different things. How does that make any sense? Do you see what I'm saying? Yeah.

Yeah, pretty much. The reason for that is because your organization should be the architecture of the machinery of building the product. That's what a company is. And yet everybody's company look exactly the same, but they all build different things. How does that make any sense? Do you see what I'm saying? Yeah.

How you make fried chicken versus how you flip burgers versus how you make Chinese fried rice is different. And so why would the machinery, why would the process be exactly the same? And so it's not sensible to me that if you look at the org charts of most companies, it all kind of looks like this.

How you make fried chicken versus how you flip burgers versus how you make Chinese fried rice is different. And so why would the machinery, why would the process be exactly the same? And so it's not sensible to me that if you look at the org charts of most companies, it all kind of looks like this.

And then you have one group that's for a business and you have another for another business, you have another for another business, and they're all kind of supposedly autonomous. And so none of that stuff makes any sense to me. It just depends on what is it that we're trying to build and what is the architecture of the company that best suits to go build it. So that's number one.

And then you have one group that's for a business and you have another for another business, you have another for another business, and they're all kind of supposedly autonomous. And so none of that stuff makes any sense to me. It just depends on what is it that we're trying to build and what is the architecture of the company that best suits to go build it. So that's number one.

In terms of information system and how do you enable collaboration, we kind of wired up like a neural network. And the way that we say is that there's a phrase in the company called mission is the boss. And so we figure out what is the mission, and we go wire up the best skills and the best teams and the best resources to achieve that mission.

In terms of information system and how do you enable collaboration, we kind of wired up like a neural network. And the way that we say is that there's a phrase in the company called mission is the boss. And so we figure out what is the mission, and we go wire up the best skills and the best teams and the best resources to achieve that mission.

And it cuts across the entire organization in a way that doesn't make any sense, but it looks a little bit like a neural network.

And it cuts across the entire organization in a way that doesn't make any sense, but it looks a little bit like a neural network.

Build Hopper or somebody else's build a system for Hopper. Somebody has built CUDA for Hopper. Somebody's job is build cuDNN for CUDA for Hopper. Somebody's job is the mission, right? So your mission is to do something.

Build Hopper or somebody else's build a system for Hopper. Somebody has built CUDA for Hopper. Somebody's job is build cuDNN for CUDA for Hopper. Somebody's job is the mission, right? So your mission is to do something.

The downside is the pressure on the leaders is fairly high. And the reason for that is because in a command and control system, the person who you report to has more power than you. And the reason why they have more power than you is because they're closer to the source of information than you are.

The downside is the pressure on the leaders is fairly high. And the reason for that is because in a command and control system, the person who you report to has more power than you. And the reason why they have more power than you is because they're closer to the source of information than you are.

In our company, the information is disseminated fairly quickly to a lot of different people, and it's usually at a team level. So for example, just now I was in our robotics meeting,

In our company, the information is disseminated fairly quickly to a lot of different people, and it's usually at a team level. So for example, just now I was in our robotics meeting,

and we're talking about certain things and we're making some decisions, and there are new college grads in a room, there's three vice presidents in a room, there's two E-staffs in a room, and at the moment that we decided together, we reasoned through some stuff, we made a decision, everybody heard it at exactly the same time. So nobody has more power than anybody else. Does it make sense?

and we're talking about certain things and we're making some decisions, and there are new college grads in a room, there's three vice presidents in a room, there's two E-staffs in a room, and at the moment that we decided together, we reasoned through some stuff, we made a decision, everybody heard it at exactly the same time. So nobody has more power than anybody else. Does it make sense?

The new college grad learned at exactly the same time as the E-staff. And so the executive staff and the leaders that work for me and myself, you earn the right to have your job based on your ability to reason through problems and helping other people succeed. And it's not because you have some privileged information that I knew the answer was 3.7 and only I knew. You know, everybody knew.

The new college grad learned at exactly the same time as the E-staff. And so the executive staff and the leaders that work for me and myself, you earn the right to have your job based on your ability to reason through problems and helping other people succeed. And it's not because you have some privileged information that I knew the answer was 3.7 and only I knew. You know, everybody knew.

In the last 30 years, I've read my fair share of business books. And as in everything you read, you're supposed to, first of all, enjoy it, right? Enjoy it, be inspired by it, but not to adopt it. That's not the whole point of these books. The whole point of these books is to share their experiences. And you're supposed to ask, what does it mean to me in my world?

In the last 30 years, I've read my fair share of business books. And as in everything you read, you're supposed to, first of all, enjoy it, right? Enjoy it, be inspired by it, but not to adopt it. That's not the whole point of these books. The whole point of these books is to share their experiences. And you're supposed to ask, what does it mean to me in my world?

And what does it mean to me in the context of what I'm going through? What does this mean to me in the environment that I'm in? And what does this mean to me in what I'm trying to achieve? And what does this mean to NVIDIA in the age of our company and the capability of our company? And so you're supposed to ask yourself, what does it mean to you?

And what does it mean to me in the context of what I'm going through? What does this mean to me in the environment that I'm in? And what does this mean to me in what I'm trying to achieve? And what does this mean to NVIDIA in the age of our company and the capability of our company? And so you're supposed to ask yourself, what does it mean to you?

And then from that point, being informed by all these different things that we're learning, we're supposed to come up with our own strategies. You know, what I just described is kind of how I go about everything. You're supposed to be inspired and learn from everybody else. And the education's free, you know? When somebody talks about a new product, you're supposed to go listen to it.

And then from that point, being informed by all these different things that we're learning, we're supposed to come up with our own strategies. You know, what I just described is kind of how I go about everything. You're supposed to be inspired and learn from everybody else. And the education's free, you know? When somebody talks about a new product, you're supposed to go listen to it.

You're not supposed to ignore it. You're supposed to go learn from it. And it could be a competitor, it could be adjacent industry, it could be nothing to do with us. The more we learn from what's happening out in the world, the better. But you're supposed to come back and ask yourself, you know, what does this mean to us?

You're not supposed to ignore it. You're supposed to go learn from it. And it could be a competitor, it could be adjacent industry, it could be nothing to do with us. The more we learn from what's happening out in the world, the better. But you're supposed to come back and ask yourself, you know, what does this mean to us?

That's right. Yeah.

That's right. Yeah.

Our journey to the data center happened, I would say, almost 17 years ago. I'm always being asked, I mean, what are the challenges that the company could see someday? And I've always felt that the fact that NVIDIA's technology is plugged into a computer and that computer has to sit next to you. because it has to be connected to a monitor.

Our journey to the data center happened, I would say, almost 17 years ago. I'm always being asked, I mean, what are the challenges that the company could see someday? And I've always felt that the fact that NVIDIA's technology is plugged into a computer and that computer has to sit next to you. because it has to be connected to a monitor.

That will limit our opportunity someday because there are only so many desktop PCs that plug a GPU into. There's only so many CRTs and the time LCDs that we could possibly drive. So the question is, wouldn't it be amazing if our computer doesn't have to be connected to the viewing device, that the separation of it made it possible for us to compute somewhere else?

That will limit our opportunity someday because there are only so many desktop PCs that plug a GPU into. There's only so many CRTs and the time LCDs that we could possibly drive. So the question is, wouldn't it be amazing if our computer doesn't have to be connected to the viewing device, that the separation of it made it possible for us to compute somewhere else?

And one of our engineers came and showed it to me one day, and it was really capturing the frame buffer, encoding it into video, and streaming it to a receiver device, separating computing from the viewing device.

And one of our engineers came and showed it to me one day, and it was really capturing the frame buffer, encoding it into video, and streaming it to a receiver device, separating computing from the viewing device.

In fact, that was when we started GFN. We knew that GFN was going to be a journey that would take a long time because you're fighting all kinds of problems, including the speed of light. Latency everywhere you look. That's right.

In fact, that was when we started GFN. We knew that GFN was going to be a journey that would take a long time because you're fighting all kinds of problems, including the speed of light. Latency everywhere you look. That's right.

GeForce Now. Yeah. Yeah, GeForce Now. We've been working on GeForce Now.

GeForce Now. Yeah. Yeah, GeForce Now. We've been working on GeForce Now.

That's right. Look at GeForce Now was NVIDIA's first data center product. And our second data center product was remote graphics, putting our GPUs in the world's enterprise data centers, which then led us to our third product, which combined CUDA plus our GPU, which became a supercomputer, which then worked towards more and more and more.

That's right. Look at GeForce Now was NVIDIA's first data center product. And our second data center product was remote graphics, putting our GPUs in the world's enterprise data centers, which then led us to our third product, which combined CUDA plus our GPU, which became a supercomputer, which then worked towards more and more and more.

And the reason why it's so important is because the disconnection between where NVIDIA's computing is done versus where it's enjoyed, if you can separate that, your market opportunity explodes. Yeah, yeah. And it was completely true. And so we're no longer limited by the physical constraints of the desktop PC sitting by your desk. And we're not limited by one GPU per person.

And the reason why it's so important is because the disconnection between where NVIDIA's computing is done versus where it's enjoyed, if you can separate that, your market opportunity explodes. Yeah, yeah. And it was completely true. And so we're no longer limited by the physical constraints of the desktop PC sitting by your desk. And we're not limited by one GPU per person.

And so it doesn't matter where it is anymore. And so that was really the great observation.

And so it doesn't matter where it is anymore. And so that was really the great observation.

Yeah, that's right. You want to pave the way to future opportunities. You can't wait until the opportunity is sitting in front of you for you to reach out for it. And so you have to anticipate. Our job as CEOs is to look around corners and to anticipate where will opportunities be someday? And even if I'm not exactly sure what and when, how do I position the company to be near it?

Yeah, that's right. You want to pave the way to future opportunities. You can't wait until the opportunity is sitting in front of you for you to reach out for it. And so you have to anticipate. Our job as CEOs is to look around corners and to anticipate where will opportunities be someday? And even if I'm not exactly sure what and when, how do I position the company to be near it?

to be just standing near under the tree, and we can do a diving catch when the apple falls. You guys know what I'm saying? Yeah. But you've got to be close enough to do the diving catch.

to be just standing near under the tree, and we can do a diving catch when the apple falls. You guys know what I'm saying? Yeah. But you've got to be close enough to do the diving catch.

Yeah. But the idea that computing will be mostly done away from the viewing device. that the vast majority of computing would be done away from the computer itself. That insight was good. In fact, cloud computing, everything about today's computing is about separation of that.

Yeah. But the idea that computing will be mostly done away from the viewing device. that the vast majority of computing would be done away from the computer itself. That insight was good. In fact, cloud computing, everything about today's computing is about separation of that.

And by putting it in a data center, we can overcome this latency problem, meaning you're not going to overcome speed of light. Speed of light end-to-end is only 120 milliseconds or something like that. It's not that long.

And by putting it in a data center, we can overcome this latency problem, meaning you're not going to overcome speed of light. Speed of light end-to-end is only 120 milliseconds or something like that. It's not that long.

Anywhere on the planet, yeah. Oh, I see.

Anywhere on the planet, yeah. Oh, I see.

Yeah, right. So if you could solve that problem, approximately something like that, I forget the number, but 70 milliseconds, 100 milliseconds, but it's not that long. And so my point is, if you could remove the obstacles everywhere else, then speed of light should be, you know, perfectly fine. And you could build data centers as large as you like, and you can do amazing things.

Yeah, right. So if you could solve that problem, approximately something like that, I forget the number, but 70 milliseconds, 100 milliseconds, but it's not that long. And so my point is, if you could remove the obstacles everywhere else, then speed of light should be, you know, perfectly fine. And you could build data centers as large as you like, and you can do amazing things.

And this little tiny device that we use as a computer or, you know, your TV as a computer, whatever computer, they can all instantly become amazing. And so that insight, you know, 15 years ago was a good one.

And this little tiny device that we use as a computer or, you know, your TV as a computer, whatever computer, they can all instantly become amazing. And so that insight, you know, 15 years ago was a good one.

Well, there are several reasons for that. First, if you want to be a data center company, building the processing chip isn't the way to do it. A data center is distinguished from a desktop computer versus a cell phone, not by the processor in it. A desktop computer in a data center uses the same CPUs, uses the same GPUs apparently, very close.

Well, there are several reasons for that. First, if you want to be a data center company, building the processing chip isn't the way to do it. A data center is distinguished from a desktop computer versus a cell phone, not by the processor in it. A desktop computer in a data center uses the same CPUs, uses the same GPUs apparently, very close.

So it's not the chip, it's not the processing chip that describes it, but it's the networking of it, it's the infrastructure of it, it's how the computing is distributed. how security is provided, how networking is done, you know, so on and so forth. And so those characteristics are associated with Mellanox, not NVIDIA.

So it's not the chip, it's not the processing chip that describes it, but it's the networking of it, it's the infrastructure of it, it's how the computing is distributed. how security is provided, how networking is done, you know, so on and so forth. And so those characteristics are associated with Mellanox, not NVIDIA.

And so the day that I concluded that really NVIDIA wants to build computers of the future and computers of the future are going to be data centers, embodied in data centers, and we want to be data center-oriented company, then we really need to get into networking, and so that was one.

And so the day that I concluded that really NVIDIA wants to build computers of the future and computers of the future are going to be data centers, embodied in data centers, and we want to be data center-oriented company, then we really need to get into networking, and so that was one.

The second thing is observation that whereas cloud computing started in hyperscale, which is about taking commodity components, a lot of users, and virtualizing many users on top of one computer, AI is really about distributed computing where one job, one training job is orchestrated across millions of processors. And so it's the inverse of hyperscale almost.

The second thing is observation that whereas cloud computing started in hyperscale, which is about taking commodity components, a lot of users, and virtualizing many users on top of one computer, AI is really about distributed computing where one job, one training job is orchestrated across millions of processors. And so it's the inverse of hyperscale almost.

I didn't learn about it until it was too late. We should have implemented all 32. But we built what we built and so we had to make the best of it. That was really an extraordinary time. Remember, Revo 120 was NV3. NV1 and NV2 were based on forward texture mapping, no triangles but curves, and it tessellated the curves.

I didn't learn about it until it was too late. We should have implemented all 32. But we built what we built and so we had to make the best of it. That was really an extraordinary time. Remember, Revo 120 was NV3. NV1 and NV2 were based on forward texture mapping, no triangles but curves, and it tessellated the curves.

And the way that you design a hyperscale computer with off-the-shelf commodity Ethernet, which is just fine for Hadoop, it's just fine for search queries, it's just fine for all of those things. But not when you're sharding a model across multiple racks. Not when you're sharding a model across, right. And so that observation says that the type of networking you want to do is not exactly Ethernet.

And the way that you design a hyperscale computer with off-the-shelf commodity Ethernet, which is just fine for Hadoop, it's just fine for search queries, it's just fine for all of those things. But not when you're sharding a model across multiple racks. Not when you're sharding a model across, right. And so that observation says that the type of networking you want to do is not exactly Ethernet.

And the way that we do networking for supercomputing is really quite ideal. And so the combination of those two ideas convinced me that Mellanox is absolutely the right company, because they're the world's leading high-performance networking company, and we worked with them in so many different areas in high-performance computing already. Plus, I really like the people.

And the way that we do networking for supercomputing is really quite ideal. And so the combination of those two ideas convinced me that Mellanox is absolutely the right company, because they're the world's leading high-performance networking company, and we worked with them in so many different areas in high-performance computing already. Plus, I really like the people.

The Israel team is world-class. We have some 3,200 people there now, and it was one of the best strategic decisions I'd ever made.

The Israel team is world-class. We have some 3,200 people there now, and it was one of the best strategic decisions I'd ever made.

I think so too, yeah. And it's so disconnected from the work that we normally do, it was surprising to everybody.

I think so too, yeah. And it's so disconnected from the work that we normally do, it was surprising to everybody.

You want to position yourself near opportunities. You don't have to be that perfect, you know. You want to position yourself near the tree. And even if you don't catch the apple before it hits the ground, so long as you're the first one to pick it up. Yeah. You want to position yourself close to the opportunities.

You want to position yourself near opportunities. You don't have to be that perfect, you know. You want to position yourself near the tree. And even if you don't catch the apple before it hits the ground, so long as you're the first one to pick it up. Yeah. You want to position yourself close to the opportunities.

And so that's kind of a lot of my work is positioning the company near opportunities and having the company having the skills to monetize each one of the steps along the way so that we can be sustainable.

And so that's kind of a lot of my work is positioning the company near opportunities and having the company having the skills to monetize each one of the steps along the way so that we can be sustainable.

Yeah, there you go. Yeah, that's a good one.

Yeah, there you go. Yeah, that's a good one.

And because we were rendering higher-level objects, we essentially avoided using Z-buffers. And we thought that that was going to be a good rendering approach, and it turns out to have been completely the wrong answer. And so what Revo Run 28 was, was a reset of our company. Now remember, at the time that we started the company in 1993, we were the only consumer 3D graphics company ever created.

And because we were rendering higher-level objects, we essentially avoided using Z-buffers. And we thought that that was going to be a good rendering approach, and it turns out to have been completely the wrong answer. And so what Revo Run 28 was, was a reset of our company. Now remember, at the time that we started the company in 1993, we were the only consumer 3D graphics company ever created.

while there are all kinds of ways to think about competition, we prefer to position ourselves in a way that serves a need that usually hasn't emerged.

while there are all kinds of ways to think about competition, we prefer to position ourselves in a way that serves a need that usually hasn't emerged.

Yeah, that's exactly right. It's our way of saying there's no market yet, but we believe there will be one. And usually when you're positioned there, everybody's trying to figure out why are you here? Right? Because when we first got into automotive, because we believe that in the future, the car is going to be largely software.

Yeah, that's exactly right. It's our way of saying there's no market yet, but we believe there will be one. And usually when you're positioned there, everybody's trying to figure out why are you here? Right? Because when we first got into automotive, because we believe that in the future, the car is going to be largely software.

And if it's going to be largely software, a really incredible computer is necessary. And so when we positioned ourselves there, most people, I still remember one of the CTOs told me, you know what, cars cannot tolerate the blue screen of death. I don't think anybody can tolerate that, but... But that doesn't change the fact that someday every car will be a software-defined car.

And if it's going to be largely software, a really incredible computer is necessary. And so when we positioned ourselves there, most people, I still remember one of the CTOs told me, you know what, cars cannot tolerate the blue screen of death. I don't think anybody can tolerate that, but... But that doesn't change the fact that someday every car will be a software-defined car.

And I think, you know, 15 years later, we're largely right. And so oftentimes there's non-consumption, and we like to navigate our company there. And by doing that, by the time the market emerges, it's very likely there aren't that many competitors shaped that way. And so we were early in PC gaming, and today NVIDIA is very large in PC gaming.

And I think, you know, 15 years later, we're largely right. And so oftentimes there's non-consumption, and we like to navigate our company there. And by doing that, by the time the market emerges, it's very likely there aren't that many competitors shaped that way. And so we were early in PC gaming, and today NVIDIA is very large in PC gaming.

We reimagined what a design workstation would be like, and today just about every workstation on the planet uses NVIDIA's technology. We reimagined how supercomputing ought to be done and who should benefit from supercomputing, that we would democratize it. And look, today, NVIDIA's accelerated computing is quite large. And we reimagined how software would be done.

We reimagined what a design workstation would be like, and today just about every workstation on the planet uses NVIDIA's technology. We reimagined how supercomputing ought to be done and who should benefit from supercomputing, that we would democratize it. And look, today, NVIDIA's accelerated computing is quite large. And we reimagined how software would be done.

And today, it's called machine learning. And how computing would be done, we call it AI. And so we reimagined these kind of things, tried to do that about a decade in advance. And so we spent about a decade in $0 billion markets. And Today, I spend a lot of time on Omniverse, and Omniverse is a classic example of a $0 billion business.

And today, it's called machine learning. And how computing would be done, we call it AI. And so we reimagined these kind of things, tried to do that about a decade in advance. And so we spent about a decade in $0 billion markets. And Today, I spend a lot of time on Omniverse, and Omniverse is a classic example of a $0 billion business.

And we were focused on transforming the PC into an accelerated PC because at the time, Windows was really a software rendered system. And so anyways, Revo 128 was a reset of our company because by the time that we realized we had gone down the wrong road, Microsoft had already rolled out DirectX. It was fundamentally incompatible with Nvidia's architecture.

And we were focused on transforming the PC into an accelerated PC because at the time, Windows was really a software rendered system. And so anyways, Revo 128 was a reset of our company because by the time that we realized we had gone down the wrong road, Microsoft had already rolled out DirectX. It was fundamentally incompatible with Nvidia's architecture.

Oftentimes, if you created the market, you ended up having what people describe as moats. Because if you build your product right and it's enabled an entire ecosystem around you to help serve that end market, you've essentially created a platform. Sometimes it's a product-based platform, sometimes it's a service-based platform, sometimes it's a technology-based platform.

Oftentimes, if you created the market, you ended up having what people describe as moats. Because if you build your product right and it's enabled an entire ecosystem around you to help serve that end market, you've essentially created a platform. Sometimes it's a product-based platform, sometimes it's a service-based platform, sometimes it's a technology-based platform.

But if you were early there and you were mindful about helping the ecosystem succeed with you, you ended up having this network of networks and all these developers and all these customers who are built around you. And that network is essentially your moat. And so, I don't love thinking about it in the context of a moat,

But if you were early there and you were mindful about helping the ecosystem succeed with you, you ended up having this network of networks and all these developers and all these customers who are built around you. And that network is essentially your moat. And so, I don't love thinking about it in the context of a moat,

And the reason for that is because you're now focused on building stuff around your castle. I tend to like thinking about things in the context of building a network. And that network is about enabling other people to enjoy the success of the final market. You know, that you're not the only company that enjoys it, but you're enjoying it with a whole bunch of other people, including me.

And the reason for that is because you're now focused on building stuff around your castle. I tend to like thinking about things in the context of building a network. And that network is about enabling other people to enjoy the success of the final market. You know, that you're not the only company that enjoys it, but you're enjoying it with a whole bunch of other people, including me.

is because from the very first day of our company, we had this architecture called UDA. It's the UDA of CUDA.

is because from the very first day of our company, we had this architecture called UDA. It's the UDA of CUDA.

That's right. The reason for that is because what we've done, what we essentially did in the beginning, even though Revo 128 only had computer graphics, the architecture described accelerators of all kinds. And we would take that architecture and developers would program to it. In fact, NVIDIA's first strategy, business strategy, was we were going to be a game console inside the PC.

That's right. The reason for that is because what we've done, what we essentially did in the beginning, even though Revo 128 only had computer graphics, the architecture described accelerators of all kinds. And we would take that architecture and developers would program to it. In fact, NVIDIA's first strategy, business strategy, was we were going to be a game console inside the PC.

And a game console needs developers, which is the reason why NVIDIA, a long time ago, one of our first employees was a developer relations person. And so it's the reason why we knew all the game developers and all the 3D developers and we knew everything.

And a game console needs developers, which is the reason why NVIDIA, a long time ago, one of our first employees was a developer relations person. And so it's the reason why we knew all the game developers and all the 3D developers and we knew everything.

Yeah, compete with Nintendo and Sega as like with PCs? In fact, the original NVIDIA architecture was called Direct Envy. Direct NVIDIA, yeah. And DirectX was an API that made it possible for operating system to directly connect hardware.

Yeah, compete with Nintendo and Sega as like with PCs? In fact, the original NVIDIA architecture was called Direct Envy. Direct NVIDIA, yeah. And DirectX was an API that made it possible for operating system to directly connect hardware.

30 competitors have already shown up, even though we were the first company at the time that we were founded. So the world was a completely different place. The question about what to do as a company strategy, at that point, I would have said that we made a whole bunch of wrong decisions. But on that day that mattered, we made a sequence of extraordinarily good decisions.

30 competitors have already shown up, even though we were the first company at the time that we were founded. So the world was a completely different place. The question about what to do as a company strategy, at that point, I would have said that we made a whole bunch of wrong decisions. But on that day that mattered, we made a sequence of extraordinarily good decisions.

In 1993, we had Direct NVIDIA, which in 1995 became, well, DirectX came out.

In 1993, we had Direct NVIDIA, which in 1995 became, well, DirectX came out.

No, but they had a lot. They did it very differently, and they did a lot of things right. We did a lot of things wrong. You were competing against Microsoft in the 90s.

No, but they had a lot. They did it very differently, and they did a lot of things right. We did a lot of things wrong. You were competing against Microsoft in the 90s.

No, it's a lot different, but I appreciate that. But we were nowhere near competing with them. If you look now, when CUDA came along, there was OpenGL, there was DirectX, but there's still another extension, if you will, and that extension is CUDA. And that CUDA extension allows a chip that got paid for running DirectX and OpenGL to create an install base for CUDA.

No, it's a lot different, but I appreciate that. But we were nowhere near competing with them. If you look now, when CUDA came along, there was OpenGL, there was DirectX, but there's still another extension, if you will, and that extension is CUDA. And that CUDA extension allows a chip that got paid for running DirectX and OpenGL to create an install base for CUDA.

And so that's the immediate strategy.

And so that's the immediate strategy.

Yeah, if you're a computing platform, everything's got to be compatible. We are the only accelerator on the planet where every single accelerator is architecturally compatible with the others. None has ever existed. There are literally a couple of hundred million, right? 250 million, 300 million installed base of active CUDA GPUs being used in the world today.

Yeah, if you're a computing platform, everything's got to be compatible. We are the only accelerator on the planet where every single accelerator is architecturally compatible with the others. None has ever existed. There are literally a couple of hundred million, right? 250 million, 300 million installed base of active CUDA GPUs being used in the world today.

And they're all architecturally compatible. How would you have a computing platform if, you know, NV30 and NV39 and NV40, they're all different? At 30 years, it's all completely compatible. And so that's the only un-negotiable rule in our company. Everything else is negotiable.

And they're all architecturally compatible. How would you have a computing platform if, you know, NV30 and NV39 and NV40, they're all different? At 30 years, it's all completely compatible. And so that's the only un-negotiable rule in our company. Everything else is negotiable.

For the record, I didn't help any of the founding CEOs that are listening. I got to tell you, while you were asking that question, what lessons would I impart? I don't know. I mean, the characteristics of successful companies and successful CEOs, I think, are fairly well described. There are a whole bunch of them. I just think starting successful companies are insanely hard.

For the record, I didn't help any of the founding CEOs that are listening. I got to tell you, while you were asking that question, what lessons would I impart? I don't know. I mean, the characteristics of successful companies and successful CEOs, I think, are fairly well described. There are a whole bunch of them. I just think starting successful companies are insanely hard.

It's just insanely hard. And when I see these amazing companies getting built, I have nothing but admiration and respect, because I just know that it's insanely hard. And I think that everybody did many similar things. There are some good, smart things that people do. There are some dumb things that you can do. But you could do all the right, smart things and still fail.

It's just insanely hard. And when I see these amazing companies getting built, I have nothing but admiration and respect, because I just know that it's insanely hard. And I think that everybody did many similar things. There are some good, smart things that people do. There are some dumb things that you can do. But you could do all the right, smart things and still fail.

You could do a whole bunch of dumb things, and I did many of them, and still succeed. So obviously, that's not exactly right. I think skills are the things that you can learn along the way. But at important moments, certain circumstances have to come together. And I do think that the market has to be one of the agents to help you succeed.

You could do a whole bunch of dumb things, and I did many of them, and still succeed. So obviously, that's not exactly right. I think skills are the things that you can learn along the way. But at important moments, certain circumstances have to come together. And I do think that the market has to be one of the agents to help you succeed.

It's not enough, obviously, because a lot of people still fail.

It's not enough, obviously, because a lot of people still fail.

And that time, 1997, was probably NVIDIA's best moment. And the reason for that was our backs were up against the wall. We were running out of time. We were running out of money. for a lot of employees running out of hope. And the question is, what do we do? Well, the first thing that we did was we decided that, look, DirectX is now here. We're not going to fight it.

And that time, 1997, was probably NVIDIA's best moment. And the reason for that was our backs were up against the wall. We were running out of time. We were running out of money. for a lot of employees running out of hope. And the question is, what do we do? Well, the first thing that we did was we decided that, look, DirectX is now here. We're not going to fight it.

Do you remember any moments where you're like... I actually thought that you starting with Revo 128 was spot on. Revo 128, as I mentioned, the number of smart decisions we made, which are smart to this day, how we design chips is exactly the same to this day. Because gosh, you know, nobody's ever done it back then.

Do you remember any moments where you're like... I actually thought that you starting with Revo 128 was spot on. Revo 128, as I mentioned, the number of smart decisions we made, which are smart to this day, how we design chips is exactly the same to this day. Because gosh, you know, nobody's ever done it back then.

And we pulled every trick in the book in a desperation because we had no other choice. Well, guess what? That's the way things ought to be done. And now everybody does it that way. Right. Everybody does it because why should you do things twice if you can do it once? Why tape out a chip seven times if you could tape it out one time, right?

And we pulled every trick in the book in a desperation because we had no other choice. Well, guess what? That's the way things ought to be done. And now everybody does it that way. Right. Everybody does it because why should you do things twice if you can do it once? Why tape out a chip seven times if you could tape it out one time, right?

And so the most efficient, the most cost-effective, the most competitive, speed is technology, right? Speed is performance. Time to market is performance. All of those things apply. So why do things twice if you could do it once? And so Revo 128 made a lot of great decisions in how we spec products, how we think about market needs and lack of, and how do we judge markets, and all of those.

And so the most efficient, the most cost-effective, the most competitive, speed is technology, right? Speed is performance. Time to market is performance. All of those things apply. So why do things twice if you could do it once? And so Revo 128 made a lot of great decisions in how we spec products, how we think about market needs and lack of, and how do we judge markets, and all of those.

Man, we made some amazingly good decisions. Yeah, we were back against the wall. We only had one more shot to do it, but...

Man, we made some amazingly good decisions. Yeah, we were back against the wall. We only had one more shot to do it, but...

Oh yeah, so for example, luck. Let's talk about luck. If Carmack hadn't decided to use acceleration, because remember, Doom was completely software rendered, and the NVIDIA philosophy was that although general purpose computing is a fabulous thing, it's going to enable software and IT and everything, we felt that there were applications that wouldn't be possible

Oh yeah, so for example, luck. Let's talk about luck. If Carmack hadn't decided to use acceleration, because remember, Doom was completely software rendered, and the NVIDIA philosophy was that although general purpose computing is a fabulous thing, it's going to enable software and IT and everything, we felt that there were applications that wouldn't be possible

Or it would be costly if it wasn't accelerated. It should be accelerated. And 3D graphics was one of them, but it wasn't the only one. And it just happens to be the first one, and a really great one. And I still remember the first times we met John, he was quite emphatic about using CPUs, and his software render was really good.

Or it would be costly if it wasn't accelerated. It should be accelerated. And 3D graphics was one of them, but it wasn't the only one. And it just happens to be the first one, and a really great one. And I still remember the first times we met John, he was quite emphatic about using CPUs, and his software render was really good.

I mean, quite frankly, if you look at Doom, the performance of Doom was really hard to achieve, even with accelerators at the time. If you didn't filter, if you didn't have to do bi-linear filtering, it did a pretty good job.

I mean, quite frankly, if you look at Doom, the performance of Doom was really hard to achieve, even with accelerators at the time. If you didn't filter, if you didn't have to do bi-linear filtering, it did a pretty good job.

Yeah, you need Carmack to program it. Exactly. It was a genius piece of code. But nonetheless, software renders did a really good job. And if he hadn't decided to go to OpenGL and accelerate for Quake, frankly, what would be the killer app that put us here?

Yeah, you need Carmack to program it. Exactly. It was a genius piece of code. But nonetheless, software renders did a really good job. And if he hadn't decided to go to OpenGL and accelerate for Quake, frankly, what would be the killer app that put us here?

And so Carmack and Sweeney, both between Unreal and Quake, created the first two killer applications for consumer 3D. And so I owe them a great deal.

And so Carmack and Sweeney, both between Unreal and Quake, created the first two killer applications for consumer 3D. And so I owe them a great deal.

Let's go figure out a way to build the best thing in the world for it. And Revo 128 is the world's first fully accelerated hardware accelerated pipeline for rendering 3D. And so the transform, the projection, every single element all the way down to the frame buffer was completely hardware accelerated. We implemented a texture cache.

Let's go figure out a way to build the best thing in the world for it. And Revo 128 is the world's first fully accelerated hardware accelerated pipeline for rendering 3D. And so the transform, the projection, every single element all the way down to the frame buffer was completely hardware accelerated. We implemented a texture cache.

Yeah. As many good ideas as we have, you don't have all the world's good ideas. And the benefit of having developers is you get to see a lot of good ideas.

Yeah. As many good ideas as we have, you don't have all the world's good ideas. And the benefit of having developers is you get to see a lot of good ideas.

And there's a couple of different areas of AI safety that's really important. Obviously, in robotics and self-driving car, there's a whole field of AI safety, and we've dedicated ourselves to functional safety and active safety and all kinds of different areas of safety. When to apply human in the loop, when is it okay for human not to be in the loop? How do you get to a point where

And there's a couple of different areas of AI safety that's really important. Obviously, in robotics and self-driving car, there's a whole field of AI safety, and we've dedicated ourselves to functional safety and active safety and all kinds of different areas of safety. When to apply human in the loop, when is it okay for human not to be in the loop? How do you get to a point where

where increasingly human doesn't have to be in the loop, but human largely in the loop. In the case of information safety, obviously bias, false information, and appreciating the rights of artists and creators, that whole area deserves a lot of attention. And you've seen some of the work that we've done.

where increasingly human doesn't have to be in the loop, but human largely in the loop. In the case of information safety, obviously bias, false information, and appreciating the rights of artists and creators, that whole area deserves a lot of attention. And you've seen some of the work that we've done.

Instead of scraping the internet, we partnered with Getty and Shutterstock to create a commercially fair way of applying artificial intelligence generated AI. In the area of large language models and the future of increasingly greater agency AI, clearly the answer is for as long as it's sensible, and I think it's going to be sensible for a long time, is human in the loop.

Instead of scraping the internet, we partnered with Getty and Shutterstock to create a commercially fair way of applying artificial intelligence generated AI. In the area of large language models and the future of increasingly greater agency AI, clearly the answer is for as long as it's sensible, and I think it's going to be sensible for a long time, is human in the loop.

The ability for an AI to self-learn and improve and change out in the wild in a digital form should be avoided. And we should collect data, we should carry the data, we should train the model, we should test the model, validate the model before we release it out in the wild again. So human is in the loop. Yep.

The ability for an AI to self-learn and improve and change out in the wild in a digital form should be avoided. And we should collect data, we should carry the data, we should train the model, we should test the model, validate the model before we release it out in the wild again. So human is in the loop. Yep.

There are a lot of different industries that have already demonstrated how to build systems that are safe and good for humanity, and obviously the way autopilot works for a plane and two-pilot system and then air traffic control. redundancy and diversity and all of the basic philosophies of designing safe systems apply as well in self-driving cars and so on and so forth.

There are a lot of different industries that have already demonstrated how to build systems that are safe and good for humanity, and obviously the way autopilot works for a plane and two-pilot system and then air traffic control. redundancy and diversity and all of the basic philosophies of designing safe systems apply as well in self-driving cars and so on and so forth.

And so I think there's a lot of models of creating safe AI and I think we need to apply them. With respect to automation, my feeling is that, and we'll see, but it is more likely that AI is going to create more jobs in the near term. The question is, what's the definition of near term? And the reason for that is the first thing that happens with productivity is prosperity.

And so I think there's a lot of models of creating safe AI and I think we need to apply them. With respect to automation, my feeling is that, and we'll see, but it is more likely that AI is going to create more jobs in the near term. The question is, what's the definition of near term? And the reason for that is the first thing that happens with productivity is prosperity.

And prosperity, when the companies get more successful, they hire more people because they want to expand into more areas. And so the question is, if you think about a company and say, okay, if we improve the productivity, then they need fewer people. Well, that's because the company has no more ideas, but that's not true for most companies.

And prosperity, when the companies get more successful, they hire more people because they want to expand into more areas. And so the question is, if you think about a company and say, okay, if we improve the productivity, then they need fewer people. Well, that's because the company has no more ideas, but that's not true for most companies.

We took the frame buffer limit to as big as physics could afford at the time. We made the biggest chip that anybody had ever imagined building. We used the fastest memories. Basically, if we built that chip, there could be nothing that could be faster.

We took the frame buffer limit to as big as physics could afford at the time. We made the biggest chip that anybody had ever imagined building. We used the fastest memories. Basically, if we built that chip, there could be nothing that could be faster.

If you become more productive and the company becomes more profitable, usually... they hire more people to expand into new areas. And so long as we believe that there are more areas to expand into, that there are more ideas in drugs, drug discovery, there are more ideas in transportation, there are more ideas in retail, there are more ideas in entertainment, that there's more ideas in technology.

If you become more productive and the company becomes more profitable, usually... they hire more people to expand into new areas. And so long as we believe that there are more areas to expand into, that there are more ideas in drugs, drug discovery, there are more ideas in transportation, there are more ideas in retail, there are more ideas in entertainment, that there's more ideas in technology.

So long as we believe that there are more ideas, the prosperity of the industry, which comes from improved productivity, results in hiring more people, more ideas. Now, you go back in history, we can fairly say that today's industry is larger than the world's industries a thousand years ago. And the reason for that is because, obviously, humans have a lot of ideas.

So long as we believe that there are more ideas, the prosperity of the industry, which comes from improved productivity, results in hiring more people, more ideas. Now, you go back in history, we can fairly say that today's industry is larger than the world's industries a thousand years ago. And the reason for that is because, obviously, humans have a lot of ideas.

And I think that there's plenty of ideas yet for prosperity and plenty of ideas that can be begat from productivity improvements, but my sense is that it's likely to generate jobs. Now, obviously, Net generation of jobs doesn't guarantee that any one human doesn't get fired. I mean, that's obviously true.

And I think that there's plenty of ideas yet for prosperity and plenty of ideas that can be begat from productivity improvements, but my sense is that it's likely to generate jobs. Now, obviously, Net generation of jobs doesn't guarantee that any one human doesn't get fired. I mean, that's obviously true.

And it's more likely that someone will lose a job to someone else, some other human that uses an AI. And not likely to an AI, but to some other human that uses an AI. And so I think the first thing that everybody should do is learn how to use AI so that they can augment their own productivity

And it's more likely that someone will lose a job to someone else, some other human that uses an AI. And not likely to an AI, but to some other human that uses an AI. And so I think the first thing that everybody should do is learn how to use AI so that they can augment their own productivity

And every company should augment their own productivity to be more productive so that they can have more prosperity, hire more people. And so I think jobs will change. My guess is that we'll actually have higher employment. We'll create more jobs. I think industries will be more productive.

And every company should augment their own productivity to be more productive so that they can have more prosperity, hire more people. And so I think jobs will change. My guess is that we'll actually have higher employment. We'll create more jobs. I think industries will be more productive.

And many of the industries that are currently suffering from lack of labor workforce is likely to use AI to get themselves off the feet and get back to growth and prosperity. So I see it a little bit differently, but I do think that jobs will be affected, and I'd encourage everybody just to learn AI.

And many of the industries that are currently suffering from lack of labor workforce is likely to use AI to get themselves off the feet and get back to growth and prosperity. So I see it a little bit differently, but I do think that jobs will be affected, and I'd encourage everybody just to learn AI.

Sequoia was the first investor in our company. Yeah, of course. Yeah.

Sequoia was the first investor in our company. Yeah, of course. Yeah.

Yeah.

Yeah.

Exactly. I just gave you exactly the same example. That in fact, productivity doesn't result in us doing less. Productivity usually results in us doing more. Everything we do will be easier. but we'll end up doing more. Because we have infinite ambition. The world has infinite ambition. And so if a company is more profitable, they tend to hire more people to do more.

Exactly. I just gave you exactly the same example. That in fact, productivity doesn't result in us doing less. Productivity usually results in us doing more. Everything we do will be easier. but we'll end up doing more. Because we have infinite ambition. The world has infinite ambition. And so if a company is more profitable, they tend to hire more people to do more.

And we also chose a cost point that is substantially higher than the highest price that we think that any of our competitors would be willing to go. If we built it right, we accelerated everything, we implemented everything in DirectX that we knew of, and we built it as large as we possibly could, then obviously nobody can build something faster than that.

And we also chose a cost point that is substantially higher than the highest price that we think that any of our competitors would be willing to go. If we built it right, we accelerated everything, we implemented everything in DirectX that we knew of, and we built it as large as we possibly could, then obviously nobody can build something faster than that.

I've never read a sci-fi book before.

I've never read a sci-fi book before.

Yeah, yeah.

Yeah, yeah.

Oh, just, you know, I watch the TV show. Okay, favorite sci-fi TV show. Star Trek's my favorite. Yeah, Star Trek's my favorite.

Oh, just, you know, I watch the TV show. Okay, favorite sci-fi TV show. Star Trek's my favorite. Yeah, Star Trek's my favorite.

V'ger's an excellent one, yeah.

V'ger's an excellent one, yeah.

Oh, it's one of my favorite cars and also favorite memories. You guys might not know this, but Lori and I got engaged Christmas one year and we drove back in my brand new Supra and we totaled it. We were this close to the end. Thank God you didn't. But nonetheless, it wasn't my fault. It wasn't the Supra's fault, but it's a mark.

Oh, it's one of my favorite cars and also favorite memories. You guys might not know this, but Lori and I got engaged Christmas one year and we drove back in my brand new Supra and we totaled it. We were this close to the end. Thank God you didn't. But nonetheless, it wasn't my fault. It wasn't the Supra's fault, but it's a mark.

I love that car. I'm driven these days for security reasons and others, but I'm driven in the Mercedes EQS. It's a great car. Nice. Yeah, great car.

I love that car. I'm driven these days for security reasons and others, but I'm driven in the Mercedes EQS. It's a great car. Nice. Yeah, great car.

Yeah, we're the central computer. Sweet.

Yeah, we're the central computer. Sweet.

Clay Christensen, I think, the series is the best. I mean, there's just no two ways about it. And the reason for that is because it's so intuitive and so sensible. It's approachable. But I read a whole bunch of them, and I read just about all of them. I really enjoyed Andy Grove's books. They're all really good.

Clay Christensen, I think, the series is the best. I mean, there's just no two ways about it. And the reason for that is because it's so intuitive and so sensible. It's approachable. But I read a whole bunch of them, and I read just about all of them. I really enjoyed Andy Grove's books. They're all really good.

Grumpy, but endearing. And what he said to me the last time as he decided to invest in our company, he says, if you lose my money, I'll kill you. Of course I did. And then over the course of the decades, the years that followed, When something is nice written about us in Mercury News, it seems like he wrote it in a crayon. You know, he'll say, good job, Don.

Grumpy, but endearing. And what he said to me the last time as he decided to invest in our company, he says, if you lose my money, I'll kill you. Of course I did. And then over the course of the decades, the years that followed, When something is nice written about us in Mercury News, it seems like he wrote it in a crayon. You know, he'll say, good job, Don.

That's right. But we observed that there was a segment of the market where people were, because at the time the PC industry was still coming up and it wasn't good enough. Everybody was clamoring for the next fastest thing. And so if your performance was 10 times higher this year than what was available, there's a whole large market of enthusiasts who we believe would have gone after it.

That's right. But we observed that there was a segment of the market where people were, because at the time the PC industry was still coming up and it wasn't good enough. Everybody was clamoring for the next fastest thing. And so if your performance was 10 times higher this year than what was available, there's a whole large market of enthusiasts who we believe would have gone after it.

You know, just write over the newspaper, just, good job, Don, and he mails it to me. And I hope I've kept them, but anyways, you could tell he's a real sweetheart, but he cares about the companies. He's a special character. Yeah, he's incredible.

You know, just write over the newspaper, just, good job, Don, and he mails it to me. And I hope I've kept them, but anyways, you could tell he's a real sweetheart, but he cares about the companies. He's a special character. Yeah, he's incredible.

There's plenty of time. Yeah, there's plenty of time. If you prioritize yourself properly and you make sure that you don't let Outlook be the controller of your time, there's plenty of time.

There's plenty of time. Yeah, there's plenty of time. If you prioritize yourself properly and you make sure that you don't let Outlook be the controller of your time, there's plenty of time.

Yeah, to do anything. Just don't do everything. Prioritize your life, make sacrifices. Don't let outlook control what you do every day. Notice I was late to our meeting. And the reason for that, by the time I looked up, I, oh my gosh, you know, Ben and Dave are waiting, you know, it's already. We got time. Yeah, exactly.

Yeah, to do anything. Just don't do everything. Prioritize your life, make sacrifices. Don't let outlook control what you do every day. Notice I was late to our meeting. And the reason for that, by the time I looked up, I, oh my gosh, you know, Ben and Dave are waiting, you know, it's already. We got time. Yeah, exactly.

No, but you have to prioritize your time really carefully and don't let outlook determine that.

No, but you have to prioritize your time really carefully and don't let outlook determine that.

I'm afraid of the same things today that I was in the very beginning of this company, which is letting the employees down. You have a lot of people who joined your company because they believe in your hopes and dreams and they've adopted it as their hopes and dreams. And you want to be right for them. You want to be successful for them.

I'm afraid of the same things today that I was in the very beginning of this company, which is letting the employees down. You have a lot of people who joined your company because they believe in your hopes and dreams and they've adopted it as their hopes and dreams. And you want to be right for them. You want to be successful for them.

You want them to be able to build a great life as well as help you build a great company and be able to build a great career. You want them to have to enjoy all of that. And these days, I want them to be able to enjoy the things I've had the benefit of enjoying and all the great success I've enjoyed. I want them to be able to enjoy all of that.

You want them to be able to build a great life as well as help you build a great company and be able to build a great career. You want them to have to enjoy all of that. And these days, I want them to be able to enjoy the things I've had the benefit of enjoying and all the great success I've enjoyed. I want them to be able to enjoy all of that.

And so I think the greatest fear is that you let them down.

And so I think the greatest fear is that you let them down.

I just, I don't change jobs. You know, if it wasn't because of Chris and Curtis convincing me to do NVIDIA, I would still be at LSI Logic today, I'm certain of it. Wow, really? Yeah, yeah. I would keep doing what I'm doing. And at the time that I was there, I was completely dedicated and focused on helping LSI Logic be the best company it could be. And I was LSI Logic's best ambassador.

I just, I don't change jobs. You know, if it wasn't because of Chris and Curtis convincing me to do NVIDIA, I would still be at LSI Logic today, I'm certain of it. Wow, really? Yeah, yeah. I would keep doing what I'm doing. And at the time that I was there, I was completely dedicated and focused on helping LSI Logic be the best company it could be. And I was LSI Logic's best ambassador.

I've got great friends to this day that I've known from LSI Logic. It's a company I loved then, I love dearly today. I know exactly why it went, the revolutionary impact it had on chip design and system design and computer design. in my estimation, one of the most important companies that ever came to Silicon Valley and changed everything about how computers were made.

I've got great friends to this day that I've known from LSI Logic. It's a company I loved then, I love dearly today. I know exactly why it went, the revolutionary impact it had on chip design and system design and computer design. in my estimation, one of the most important companies that ever came to Silicon Valley and changed everything about how computers were made.

It put me in the epicenter of some of the most important events in computer industry. It led me to meeting Chris and Curtis and Andy Bechtolsheim and John Rubenstein and some of the most important people in the world. And Frank that I was with the other day and just, I mean, the list goes on. And so LSI Logic was really important to me and I would still be there.

It put me in the epicenter of some of the most important events in computer industry. It led me to meeting Chris and Curtis and Andy Bechtolsheim and John Rubenstein and some of the most important people in the world. And Frank that I was with the other day and just, I mean, the list goes on. And so LSI Logic was really important to me and I would still be there.

Who knows what LSI Logic would have become if I were still there, right? And so that's kind of how my mind works.

Who knows what LSI Logic would have become if I were still there, right? And so that's kind of how my mind works.

And we were absolutely right that the PC industry had a substantially large enthusiast market that would buy the best of everything. To this day, it kind of remains true. And for certain segments of the market where the technology is never good enough, like 3D graphics, when we chose the right technology, 3D graphics is never good enough.

And we were absolutely right that the PC industry had a substantially large enthusiast market that would buy the best of everything. To this day, it kind of remains true. And for certain segments of the market where the technology is never good enough, like 3D graphics, when we chose the right technology, 3D graphics is never good enough.

Yeah, exactly. I mean, I might be doing the same thing I'm doing today.

Yeah, exactly. I mean, I might be doing the same thing I'm doing today.

But until I'm fired, this is my last job. I love it.

But until I'm fired, this is my last job. I love it.

transistors and logical gates and chip functionality in high-level languages. that by raising the level of abstraction and what is now called high-level design, it was coined by Harvey Jones who's on NVIDIA's board and I met him way back in the early days of Synopsys. But during that time, there was this belief that you can express chip design in high-level languages.

transistors and logical gates and chip functionality in high-level languages. that by raising the level of abstraction and what is now called high-level design, it was coined by Harvey Jones who's on NVIDIA's board and I met him way back in the early days of Synopsys. But during that time, there was this belief that you can express chip design in high-level languages.

By doing so, you could take advantage of optimizing compilers and optimization logic and tools and be a lot more productive. That logic was so sensible to me, and I was 21 years old at the time, and I wanted to pursue that vision. Frankly, that idea happened in machine learning. It happened in software programming.

By doing so, you could take advantage of optimizing compilers and optimization logic and tools and be a lot more productive. That logic was so sensible to me, and I was 21 years old at the time, and I wanted to pursue that vision. Frankly, that idea happened in machine learning. It happened in software programming.

I want to see it happen in digital biology so that we can think about biology in a much higher level language. Probably a large language model would be the way to make it representable. That transition was so revolutionary, I thought that was the best thing that ever happened to the industry, and I was really happy to be part of it, and I was at ground zero.

I want to see it happen in digital biology so that we can think about biology in a much higher level language. Probably a large language model would be the way to make it representable. That transition was so revolutionary, I thought that was the best thing that ever happened to the industry, and I was really happy to be part of it, and I was at ground zero.

And so I saw one industry change revolutionize another industry. And if not for LSI Logic doing the work that it did, Synopsys, shortly after, then why would the computer industry be where it is today? Yeah, it's really, really terrific. I was at the right place at the right time to see all that.

And so I saw one industry change revolutionize another industry. And if not for LSI Logic doing the work that it did, Synopsys, shortly after, then why would the computer industry be where it is today? Yeah, it's really, really terrific. I was at the right place at the right time to see all that.

Yeah.

Yeah.

I didn't know how to write a business plan.

I didn't know how to write a business plan.

No, no, no. It turns out that making a financial forecast that nobody knows is going to be right or wrong turns out not to be that important. But the important things that a business plan probably could have teased out, I think that the art of writing a business plan ought to be much, much shorter. And it forces you to condense, you know, what is the true problem you're trying to solve?

No, no, no. It turns out that making a financial forecast that nobody knows is going to be right or wrong turns out not to be that important. But the important things that a business plan probably could have teased out, I think that the art of writing a business plan ought to be much, much shorter. And it forces you to condense, you know, what is the true problem you're trying to solve?

What is the unmet need that you believe will emerge? And what is it that you're going to do that is sufficiently hard that when everybody else finds out it's a good idea, they're not going to swarm it and, you know, make you obsolete? And so it has to be sufficiently hard to do. There are a whole bunch of other skills that are involved in just, you know,

What is the unmet need that you believe will emerge? And what is it that you're going to do that is sufficiently hard that when everybody else finds out it's a good idea, they're not going to swarm it and, you know, make you obsolete? And so it has to be sufficiently hard to do. There are a whole bunch of other skills that are involved in just, you know,

And we call it back then, 3D gives us sustainable technology opportunity because it's never good enough. And so your technology can keep getting better. We chose that. We also made the decision to use this technology called emulation. There was a company called Icos. And on the day that I called them, they were just shutting the company down because they had no customers.

And we call it back then, 3D gives us sustainable technology opportunity because it's never good enough. And so your technology can keep getting better. We chose that. We also made the decision to use this technology called emulation. There was a company called Icos. And on the day that I called them, they were just shutting the company down because they had no customers.

product and positioning and pricing and go to market and all that kind of stuff. But those are skills and you can learn those things easily. The stuff that is really, really hard is the essence of what I described. And I did that okay, but I had no idea how to write the business plan. And I was fortunate that Wolf Corrigan was so pleased with me and the work that I did when I was at LSLogic.

product and positioning and pricing and go to market and all that kind of stuff. But those are skills and you can learn those things easily. The stuff that is really, really hard is the essence of what I described. And I did that okay, but I had no idea how to write the business plan. And I was fortunate that Wolf Corrigan was so pleased with me and the work that I did when I was at LSLogic.

He called up Don Valentine and told Don, you know, invest in this kid and he's going to come your way. And so I was, you know, I was set up for success from that moment and got us on the ground. As long as he didn't lose the money. I think Sequoia did okay. I think we probably are one of the best investments they've ever made.

He called up Don Valentine and told Don, you know, invest in this kid and he's going to come your way. And so I was, you know, I was set up for success from that moment and got us on the ground. As long as he didn't lose the money. I think Sequoia did okay. I think we probably are one of the best investments they've ever made.

The VC partner is still on the board, Mark Stevens.

The VC partner is still on the board, Mark Stevens.

Yeah. Yeah. All these years. The two founding VCs are still on the board. Sutter Hill and Sequoia? Yeah, Tench Cox and Mark Stevens. I don't think that ever happens.

Yeah. Yeah. All these years. The two founding VCs are still on the board. Sutter Hill and Sequoia? Yeah, Tench Cox and Mark Stevens. I don't think that ever happens.

Amazing. We are singular in that circumstance, I believe.

Amazing. We are singular in that circumstance, I believe.

they've added value this whole time uh been inspiring this whole time uh uh gave great wisdom and and uh great support uh but they they also uh they haven't killed you yet no not yet but they've been entertained you know by the company inspired by the company and enriched by the company and so they stayed with it and i'm really grateful well in that being our final question for you it's 2023 30 years anniversary of the founding of nvidia

they've added value this whole time uh been inspiring this whole time uh uh gave great wisdom and and uh great support uh but they they also uh they haven't killed you yet no not yet but they've been entertained you know by the company inspired by the company and enriched by the company and so they stayed with it and i'm really grateful well in that being our final question for you it's 2023 30 years anniversary of the founding of nvidia

I wouldn't do it. I know. And the reason for that is really quite simple, ignoring the company that we would start. First of all, I'm not exactly sure. The reason why I wouldn't do it, and it goes back to why it's so hard, is building a company and building a video turned out to have been a million times harder than I expected it to be, any of us expected it to be.

I wouldn't do it. I know. And the reason for that is really quite simple, ignoring the company that we would start. First of all, I'm not exactly sure. The reason why I wouldn't do it, and it goes back to why it's so hard, is building a company and building a video turned out to have been a million times harder than I expected it to be, any of us expected it to be.

And at that time, if we realized the pain and suffering and just how vulnerable you're gonna feel, and the challenges that you're going to endure, the embarrassment and the shame and the list of all the things that go wrong, I don't think anybody would start a company. Nobody in their right mind would do it. And I think that that's kind of the superpower of a entrepreneur.

And at that time, if we realized the pain and suffering and just how vulnerable you're gonna feel, and the challenges that you're going to endure, the embarrassment and the shame and the list of all the things that go wrong, I don't think anybody would start a company. Nobody in their right mind would do it. And I think that that's kind of the superpower of a entrepreneur.

They don't know how hard it is. And they only ask themselves, how hard can it be? And to this day, I trick my brain into thinking, how hard can it be? Because you have to. Still, when you wake up in the morning. Yeah, how hard can it be? Everything that we're doing, how hard can it be? Omniverse, how hard can it be?

They don't know how hard it is. And they only ask themselves, how hard can it be? And to this day, I trick my brain into thinking, how hard can it be? Because you have to. Still, when you wake up in the morning. Yeah, how hard can it be? Everything that we're doing, how hard can it be? Omniverse, how hard can it be?

No, I'm still young.

No, I'm still young.

The trick is still working. The trick is still working. I'm still enjoying myself immensely and I'm adding a little bit of value, but that's really the trick of an entrepreneur. You have to get yourself to believe that it's not that hard. because it's way harder than you think.

The trick is still working. The trick is still working. I'm still enjoying myself immensely and I'm adding a little bit of value, but that's really the trick of an entrepreneur. You have to get yourself to believe that it's not that hard. because it's way harder than you think.

And I said, hey, look, I'll buy what you have inventory and no promises are necessary. And the reason why we needed that emulator is because if you figure out how much money that we have, if we taped out a chip and we got it back from the fab and we started working on our software,

And I said, hey, look, I'll buy what you have inventory and no promises are necessary. And the reason why we needed that emulator is because if you figure out how much money that we have, if we taped out a chip and we got it back from the fab and we started working on our software,

And so if I go taking all of my knowledge now and I go back and I said, I'm going to endure that whole journey again, I think it's too much. It is just too much.

And so if I go taking all of my knowledge now and I go back and I said, I'm going to endure that whole journey again, I think it's too much. It is just too much.

Family and friends and all the colleagues we have here. I'm surrounded by people who've been here for 30 years. Chris has been here for 30 years and Jeff Fisher has been here 30 years. Dwight's been here 30 years and Jonah and Brian have been here, you know, 25 some years and probably longer than that. And, you know, Joe Greco has been here 30 years.

Family and friends and all the colleagues we have here. I'm surrounded by people who've been here for 30 years. Chris has been here for 30 years and Jeff Fisher has been here 30 years. Dwight's been here 30 years and Jonah and Brian have been here, you know, 25 some years and probably longer than that. And, you know, Joe Greco has been here 30 years.

I'm surrounded by these people that never one time gave up and they never one time gave up on me. And that's the entire ball of wax, you know, and to be able to go home and have your family be fully committed to everything that you're trying to do and thick or thin, they're proud of you and proud of the company and You kind of need that. You need the unwavering support of people around you.

I'm surrounded by these people that never one time gave up and they never one time gave up on me. And that's the entire ball of wax, you know, and to be able to go home and have your family be fully committed to everything that you're trying to do and thick or thin, they're proud of you and proud of the company and You kind of need that. You need the unwavering support of people around you.

You know, Jim Gaithers and, you know, the Tench Coxes and Mark Stevens and, you know, Harvey Jones and all the early people of our company, the Bill Millers. They not one time gave up on the company and us and And you need that. Not kind of need that, you need that.

You know, Jim Gaithers and, you know, the Tench Coxes and Mark Stevens and, you know, Harvey Jones and all the early people of our company, the Bill Millers. They not one time gave up on the company and us and And you need that. Not kind of need that, you need that.

And I'm pretty sure that almost every successful company and entrepreneurs that have gone through some difficult challenges, they had that support system around them.

And I'm pretty sure that almost every successful company and entrepreneurs that have gone through some difficult challenges, they had that support system around them.

particularly amplified on these dimensions, right?

particularly amplified on these dimensions, right?

so much support yeah yeah it is incredible and you hate that any of that stuff happened and and most of you you know most of it is is out of your control but you know 80 fall it it's an extraordinary thing no matter how you look at it and

so much support yeah yeah it is incredible and you hate that any of that stuff happened and and most of you you know most of it is is out of your control but you know 80 fall it it's an extraordinary thing no matter how you look at it and

I forget exactly, but I mean, we traded down at about a couple of two, three billion dollars in market value for a while because of the decision we made in going into CUDA and all that work. And your belief system has to be really, really strong. You know, you have to really, really believe it and really, really want it. Otherwise, it's just too much to endure.

I forget exactly, but I mean, we traded down at about a couple of two, three billion dollars in market value for a while because of the decision we made in going into CUDA and all that work. And your belief system has to be really, really strong. You know, you have to really, really believe it and really, really want it. Otherwise, it's just too much to endure.

I mean, because, you know, everybody's questioning you and employees aren't questioning you, but employees have questions.

I mean, because, you know, everybody's questioning you and employees aren't questioning you, but employees have questions.

People outside are questioning you. And it's a little embarrassing. It's like, you know, when your stock price gets hit, it's embarrassing no matter how you think about it. And it's hard to explain, you know. And so there's no good answers to any of that stuff. CEOs are human and companies are built of humans and these challenges are hard to endure.

People outside are questioning you. And it's a little embarrassing. It's like, you know, when your stock price gets hit, it's embarrassing no matter how you think about it. And it's hard to explain, you know. And so there's no good answers to any of that stuff. CEOs are human and companies are built of humans and these challenges are hard to endure.

By the time that we found all the bugs, because we did the software, then we taped out the chip again, well, we would have been out of business already. And your competitors would have caught up. Well, not to mention we would have been out of business.

By the time that we found all the bugs, because we did the software, then we taped out the chip again, well, we would have been out of business already. And your competitors would have caught up. Well, not to mention we would have been out of business.

Yeah. The thing to keep in mind is at all times, what is the market opportunity that you're engaging? And that informs your size. I was told a long time ago that NVIDIA can never be larger than a billion dollars. Obviously, it's an underestimation, underimagination of the size of the opportunity. It is the case that no chip company can ever be so big.

Yeah. The thing to keep in mind is at all times, what is the market opportunity that you're engaging? And that informs your size. I was told a long time ago that NVIDIA can never be larger than a billion dollars. Obviously, it's an underestimation, underimagination of the size of the opportunity. It is the case that no chip company can ever be so big.

And so, but if you're not a chip company, then why is that applied to you? And this is the extraordinary thing about technology right now, is technology is a tool, and it's only so large. What's unique about our current circumstance today is is that we're in the manufacturing of intelligence. We're in the manufacturing of work world. That's AI.

And so, but if you're not a chip company, then why is that applied to you? And this is the extraordinary thing about technology right now, is technology is a tool, and it's only so large. What's unique about our current circumstance today is is that we're in the manufacturing of intelligence. We're in the manufacturing of work world. That's AI.

And the world of tasks, doing work, productive, generative AI work, generative intelligent work, that market size is enormous. It's measured in trillions. One way to think about that is if you built a chip for a car, how many cars are there and how many chips would they consume? That's one way to think about that.

And the world of tasks, doing work, productive, generative AI work, generative intelligent work, that market size is enormous. It's measured in trillions. One way to think about that is if you built a chip for a car, how many cars are there and how many chips would they consume? That's one way to think about that.

However, if you build a system that, whenever needed, assisted in the driving of the car, what's the value of an autonomous chauffeur every now and then? And so now the market, obviously the problem becomes much larger, the opportunity becomes larger. You know, what would it be like if we were to magically conjure up a chauffeur for everybody who has a car? And, you know, how big is that market?

However, if you build a system that, whenever needed, assisted in the driving of the car, what's the value of an autonomous chauffeur every now and then? And so now the market, obviously the problem becomes much larger, the opportunity becomes larger. You know, what would it be like if we were to magically conjure up a chauffeur for everybody who has a car? And, you know, how big is that market?

And obviously, that's a much, much larger market. And so the technology industry is at the, you know, where what we've discovered, what NVIDIA has discovered and what some of them discovered, is that by separating ourselves from being a chip company, But building on top of a chip and you're now in the ad company, the market opportunity has grown by probably a thousand times.

And obviously, that's a much, much larger market. And so the technology industry is at the, you know, where what we've discovered, what NVIDIA has discovered and what some of them discovered, is that by separating ourselves from being a chip company, But building on top of a chip and you're now in the ad company, the market opportunity has grown by probably a thousand times.

Don't be surprised if technology companies become much larger in the future because what you produce is something very different. And that's kind of the way to think about how large can your opportunity, how large can you be. It has everything to do with the size of the opportunity.

Don't be surprised if technology companies become much larger in the future because what you produce is something very different. And that's kind of the way to think about how large can your opportunity, how large can you be. It has everything to do with the size of the opportunity.

So if you're going to be out of business anyways, that plan obviously wasn't the plan. The plan that companies normally go through, which is build the chip, write the software, fix the bugs, tape out the new chip, so on and so forth, that method wasn't going to work.

So if you're going to be out of business anyways, that plan obviously wasn't the plan. The plan that companies normally go through, which is build the chip, write the software, fix the bugs, tape out the new chip, so on and so forth, that method wasn't going to work.

And so the question is, if we only had six months and you get to tape out just one time, then obviously you're going to tape out a perfect chip. So I remember having conversation with our leaders and they said, but Jensen, how do you know it's going to be perfect? And I said, I know it's going to be perfect because if it's not, we'll be out of business. And so let's make it perfect.

And so the question is, if we only had six months and you get to tape out just one time, then obviously you're going to tape out a perfect chip. So I remember having conversation with our leaders and they said, but Jensen, how do you know it's going to be perfect? And I said, I know it's going to be perfect because if it's not, we'll be out of business. And so let's make it perfect.

We get one shot. We essentially virtually prototyped the chip by buying this emulator. And Dwight and the software team wrote our software, the entire stack, and ran it on this emulator and just sat in the lab waiting for Windows to paint. It was like 60 seconds per frame or something like that. Oh, easily. I actually think that it was an hour per frame, something like that.

We get one shot. We essentially virtually prototyped the chip by buying this emulator. And Dwight and the software team wrote our software, the entire stack, and ran it on this emulator and just sat in the lab waiting for Windows to paint. It was like 60 seconds per frame or something like that. Oh, easily. I actually think that it was an hour per frame, something like that.

And so we would just sit there and watch it paint. And so on the day that we decided to tape out, I assumed that the chip was perfect. And everything that we could have tested, we tested in advance and told everybody this is it. We're going to tape out the chip. It's going to be perfect. Well, if you're going to tape out a chip and you know it's perfect, then what else would you do?

And so we would just sit there and watch it paint. And so on the day that we decided to tape out, I assumed that the chip was perfect. And everything that we could have tested, we tested in advance and told everybody this is it. We're going to tape out the chip. It's going to be perfect. Well, if you're going to tape out a chip and you know it's perfect, then what else would you do?

That's actually the good question. If you knew that you hit enter, you taped out a chip, and you knew it was going to be perfect, then what else would you do? Well, the answer, obviously, go to production. And marketing blitz.

That's actually the good question. If you knew that you hit enter, you taped out a chip, and you knew it was going to be perfect, then what else would you do? Well, the answer, obviously, go to production. And marketing blitz.

Just start, kick everything off, kick everything off because you got a perfect chip. And so we got it in our head that we have a perfect chip.

Just start, kick everything off, kick everything off because you got a perfect chip. And so we got it in our head that we have a perfect chip.

No, everybody was clear we had no shot. Not doing it would be crazy. Because the last- Otherwise, you might as well go home. Yeah, you're going to do our business anyways. So anything aside from that is crazy. So it seemed like a fairly logical thing. And quite frankly, right now, as I'm describing it, you're probably thinking, yeah, it's pretty sensible.

No, everybody was clear we had no shot. Not doing it would be crazy. Because the last- Otherwise, you might as well go home. Yeah, you're going to do our business anyways. So anything aside from that is crazy. So it seemed like a fairly logical thing. And quite frankly, right now, as I'm describing it, you're probably thinking, yeah, it's pretty sensible.

Yeah. And so we take that out and went directly to production.

Yeah. And so we take that out and went directly to production.

No. How do you think about that? No, no. When you push your chips in, I know it's going to work. Notice, we assume that we taped out a perfect chip. The reason why we taped out a perfect chip is because we emulated the whole chip before we taped it out. We developed the entire software stack. We ran QA on all the drivers and all the software. We ran all the games we had.

No. How do you think about that? No, no. When you push your chips in, I know it's going to work. Notice, we assume that we taped out a perfect chip. The reason why we taped out a perfect chip is because we emulated the whole chip before we taped it out. We developed the entire software stack. We ran QA on all the drivers and all the software. We ran all the games we had.

We ran every VGA application we had. And so when you push your chips in, what you're really doing is when you bet the farm, you're saying, I'm going to take everything in the future, all the risky things, and I'm going to pull it in advance. And that is probably the lesson. And to this day, everything that we can prefetch, everything in the future that we can simulate today, we prefetch it.

We ran every VGA application we had. And so when you push your chips in, what you're really doing is when you bet the farm, you're saying, I'm going to take everything in the future, all the risky things, and I'm going to pull it in advance. And that is probably the lesson. And to this day, everything that we can prefetch, everything in the future that we can simulate today, we prefetch it.

And so we were already playing with the concept of how do we create an abstraction layer above our chip that is expressible in a higher-level language and higher-level expression? And how can we use our GPU for things like CT reconstruction, image processing? We were already down that path.

And so we were already playing with the concept of how do we create an abstraction layer above our chip that is expressible in a higher-level language and higher-level expression? And how can we use our GPU for things like CT reconstruction, image processing? We were already down that path.

there were some positive feedback and some intuitive positive feedback that we think that general purpose computing could be possible. And if you just looked at the pipeline of a programmable shader, it is a processor and is highly parallel and it is massively threaded and it is the only processor in the world that does that.

there were some positive feedback and some intuitive positive feedback that we think that general purpose computing could be possible. And if you just looked at the pipeline of a programmable shader, it is a processor and is highly parallel and it is massively threaded and it is the only processor in the world that does that.

And so there were a lot of characteristics about programmable shading that would suggest that CUDA has a great opportunity to succeed.

And so there were a lot of characteristics about programmable shading that would suggest that CUDA has a great opportunity to succeed.

I guess yes and no. You know, when we saw deep learning, when we saw AlexNet and realized its incredible effectiveness in computer vision, we had the good sense, if you will, to go back to first principles and ask, you know, what is it about this thing that made it so successful?

I guess yes and no. You know, when we saw deep learning, when we saw AlexNet and realized its incredible effectiveness in computer vision, we had the good sense, if you will, to go back to first principles and ask, you know, what is it about this thing that made it so successful?

When a new software technology, a new algorithm comes along and somehow leapfrogs 30 years of computer vision work, you have to take a step back and ask yourself, but why? And fundamentally, is it scalable? And if it's scalable, what other problems can it solve? And there were several observations that we made.

When a new software technology, a new algorithm comes along and somehow leapfrogs 30 years of computer vision work, you have to take a step back and ask yourself, but why? And fundamentally, is it scalable? And if it's scalable, what other problems can it solve? And there were several observations that we made.

The first observation, of course, is that if you have a whole lot of example data, you could teach this function to make predictions. Well, what we've basically done is discovered a universal function approximator, because the dimensionality could be as high as you want it to be.

The first observation, of course, is that if you have a whole lot of example data, you could teach this function to make predictions. Well, what we've basically done is discovered a universal function approximator, because the dimensionality could be as high as you want it to be.

And because each layer is trained one layer at a time, there's no reason why you can't make very, very deep neural networks. Okay, so now you just reason your way through, right? Okay, so now I go back to 12 years ago. You could just imagine the reasoning I'm going through in my head, that we've discovered a universal function approximator.

And because each layer is trained one layer at a time, there's no reason why you can't make very, very deep neural networks. Okay, so now you just reason your way through, right? Okay, so now I go back to 12 years ago. You could just imagine the reasoning I'm going through in my head, that we've discovered a universal function approximator.

In fact, we might have discovered, with a couple more technologies, a universal computer.

In fact, we might have discovered, with a couple more technologies, a universal computer.

Yeah. The reason for that is because we were already working on computer vision at the time, and we were trying to get CUDA to be a good computer vision system, or most of the algorithms that were created for computer vision aren't a good fit for CUDA. So what we're sitting there trying to figure it out, all of a sudden AlexNet shows up. So that was incredibly intriguing.

Yeah. The reason for that is because we were already working on computer vision at the time, and we were trying to get CUDA to be a good computer vision system, or most of the algorithms that were created for computer vision aren't a good fit for CUDA. So what we're sitting there trying to figure it out, all of a sudden AlexNet shows up. So that was incredibly intriguing.

It's so effective that it makes you take a step back and ask yourself, why is that happening? So by the time that you reason your way through this, you go, well, what are the kind of problems in a world where a universal function approximator can solve, right? Well, we know that most of our algorithms start from principled sciences, okay?

It's so effective that it makes you take a step back and ask yourself, why is that happening? So by the time that you reason your way through this, you go, well, what are the kind of problems in a world where a universal function approximator can solve, right? Well, we know that most of our algorithms start from principled sciences, okay?

You want to understand the causality, and from the causality, you create a simulation algorithm that allows us to scale. Well, for a lot of problems, we kind of don't care about the causality. We just care about the predictability of it. Like, do I really care for what reason you prefer this toothpaste over that? I don't really care the causality.

You want to understand the causality, and from the causality, you create a simulation algorithm that allows us to scale. Well, for a lot of problems, we kind of don't care about the causality. We just care about the predictability of it. Like, do I really care for what reason you prefer this toothpaste over that? I don't really care the causality.

I just want to know that this is the one you would have predicted. Do I really care that the fundamental cause of somebody who buys a hot dog buys ketchup and mustard? It doesn't really matter. It only matters that I can predict it. It applies to predicting movies, predicting music. It applies to predicting, quite frankly, weather. We understand thermodynamics.

I just want to know that this is the one you would have predicted. Do I really care that the fundamental cause of somebody who buys a hot dog buys ketchup and mustard? It doesn't really matter. It only matters that I can predict it. It applies to predicting movies, predicting music. It applies to predicting, quite frankly, weather. We understand thermodynamics.

We understand radiation from the sun. We understand cloud effects. We understand oceanic effects. We understand all these different things. We just want to know whether we should wear a sweater or not. Isn't that right? And so causality for a lot of problems in the world doesn't matter. We just want to emulate the system and predict the outcome.

We understand radiation from the sun. We understand cloud effects. We understand oceanic effects. We understand all these different things. We just want to know whether we should wear a sweater or not. Isn't that right? And so causality for a lot of problems in the world doesn't matter. We just want to emulate the system and predict the outcome.