Nathan Lambert

caches which are shared between more compute elements then you have like memory right like hbm or dram like ddr memory or whatever it is and that's shared between the whole chip and then you can have you know pools of memory that are shared between many chips right and then storage and it keep you keep zoning out right the access latency across data centers across within the data center within a chip is different so like you're obviously always you're always going to have different programming paradigms for this it's not going to be easy programming this stuff is going to be hard maybe i can help right

caches which are shared between more compute elements then you have like memory right like hbm or dram like ddr memory or whatever it is and that's shared between the whole chip and then you can have you know pools of memory that are shared between many chips right and then storage and it keep you keep zoning out right the access latency across data centers across within the data center within a chip is different so like you're obviously always you're always going to have different programming paradigms for this it's not going to be easy programming this stuff is going to be hard maybe i can help right

caches which are shared between more compute elements then you have like memory right like hbm or dram like ddr memory or whatever it is and that's shared between the whole chip and then you can have you know pools of memory that are shared between many chips right and then storage and it keep you keep zoning out right the access latency across data centers across within the data center within a chip is different so like you're obviously always you're always going to have different programming paradigms for this it's not going to be easy programming this stuff is going to be hard maybe i can help right

um, you know, with programming this, but the, the, the way to think about it is that like, there is, there, there's sort of like the more elements you add to a task, you, you don't gain, you don't get strong scaling, right? If I double the number of chips, I don't get two X the performance, right? This is just like a reality of computing, uh, cause there's inefficiencies.

um, you know, with programming this, but the, the, the way to think about it is that like, there is, there, there's sort of like the more elements you add to a task, you, you don't gain, you don't get strong scaling, right? If I double the number of chips, I don't get two X the performance, right? This is just like a reality of computing, uh, cause there's inefficiencies.

um, you know, with programming this, but the, the, the way to think about it is that like, there is, there, there's sort of like the more elements you add to a task, you, you don't gain, you don't get strong scaling, right? If I double the number of chips, I don't get two X the performance, right? This is just like a reality of computing, uh, cause there's inefficiencies.

Um, and there's a lot of interesting work being done to make it not You know, to make it more linear, whether it's making the chips more networked together more tightly or, you know, cool programming models or cool algorithmic things that you can do on the model side. Right.

Um, and there's a lot of interesting work being done to make it not You know, to make it more linear, whether it's making the chips more networked together more tightly or, you know, cool programming models or cool algorithmic things that you can do on the model side. Right.

Um, and there's a lot of interesting work being done to make it not You know, to make it more linear, whether it's making the chips more networked together more tightly or, you know, cool programming models or cool algorithmic things that you can do on the model side. Right.

DeepSeq did some of these really cool innovations because they were limited on interconnect, but they still needed to parallelize. Right. Like all sorts of, you know, everyone's always doing stuff. Google's got a bunch of work and everyone's got a bunch of work about this. That stuff is super exciting on the model and workload and innovation side. Right.

DeepSeq did some of these really cool innovations because they were limited on interconnect, but they still needed to parallelize. Right. Like all sorts of, you know, everyone's always doing stuff. Google's got a bunch of work and everyone's got a bunch of work about this. That stuff is super exciting on the model and workload and innovation side. Right.

DeepSeq did some of these really cool innovations because they were limited on interconnect, but they still needed to parallelize. Right. Like all sorts of, you know, everyone's always doing stuff. Google's got a bunch of work and everyone's got a bunch of work about this. That stuff is super exciting on the model and workload and innovation side. Right.

Hardware, solid state transformers are interesting for the power side. There's all sorts of stuff on batteries and stuff.

Hardware, solid state transformers are interesting for the power side. There's all sorts of stuff on batteries and stuff.

Hardware, solid state transformers are interesting for the power side. There's all sorts of stuff on batteries and stuff.

There's all sorts of stuff on, you know, I think when you look at, if you look at every layer of the compute stack, right, whether it goes from lithography and etch all the way to like fabrication, to like optics, to networking, to power, to transformers, to cooling, to, you know, networking, and you just go on up and up and up and up the stack, you know, even air conditioners for data centers are like innovating.

There's all sorts of stuff on, you know, I think when you look at, if you look at every layer of the compute stack, right, whether it goes from lithography and etch all the way to like fabrication, to like optics, to networking, to power, to transformers, to cooling, to, you know, networking, and you just go on up and up and up and up the stack, you know, even air conditioners for data centers are like innovating.

There's all sorts of stuff on, you know, I think when you look at, if you look at every layer of the compute stack, right, whether it goes from lithography and etch all the way to like fabrication, to like optics, to networking, to power, to transformers, to cooling, to, you know, networking, and you just go on up and up and up and up the stack, you know, even air conditioners for data centers are like innovating.

right? Like it's like, there's like copper cables are innovating, right? Like you wouldn't think it, but copper cables, like are, there's some innovations happening there with like the density of how you can pack them. And like, it's like all of these layers of the stack, all the way up to the models, human progress is at a pace that's never been seen before.

right? Like it's like, there's like copper cables are innovating, right? Like you wouldn't think it, but copper cables, like are, there's some innovations happening there with like the density of how you can pack them. And like, it's like all of these layers of the stack, all the way up to the models, human progress is at a pace that's never been seen before.