Show notes
In this episode of CXO Bytes, Sanjay Podder is hosted by Beliz Kudat to talk about the dual role of technology in driving sustainability while also contributing to environmental challenges. They explore how businesses can integrate sustainable strategies into their technology operations to minimize carbon footprints, optimize data center energy consumption, and leverage tools like AI and cloud solutions responsibly. Sanjay highlights actionable techniques such as carbon-aware scheduling, efficient coding practices, and emerging tools to measure the energy impact of AI. The discussion also emphasizes the business value of sustainability, including improved ESG scores, employee attraction, and outperforming competitors in shareholder returns, making sustainable technology a critical strategic imperative for organizations.Learn more about our people:Sanjay Podder: LinkedInBeliz Kudat: LinkedInFind out more about the GSF:The Green Software Foundation Website Sign up to the Green Software Foundation NewsletterResources:Key Findings Data Centres Metered Electricity Consumption 2023 - Central Statistics Office) [10:55]Carbon Aware SDK [12:54]CarbonCloud [15:40]Impact Framework 15:56]If you enjoyed this episode then please either:Follow, rate, and review on Apple PodcastsFollow and rate on SpotifyWatch our videos on The Green Software Foundation YouTube Channel!Connect with us on Twitter, Github and LinkedIn!TRANSCRIPT BELOW: Sanjay Podder: Hello and welcome to CXO Bytes, a podcast brought to you by the Green Software Foundation and dedicated to supporting chiefs of information, technology, sustainability, and AI as they aim to shape a sustainable future through green software. We will uncover the strategies and a big green move that's helped drive results for business and for the planet.I am your host, Sanjay Podder.Beliz Kudat: Okay, Sanjay, welcome to our business summit. Sanjay Poddar: Thank you so much for having me today. My pleasure. Beliz Kudat: It's a pleasure having you. So, you know, in today's rapidly developing digital technologies and this digital transformation, a significant dilemma arises, especially for sustainability. And on one hand, these technologies offer substantial, huge potential to address environmental issues.And on the other hand, there exists an entire substantial resource consumption. So first, we'd like to start by asking your perspective on this and how can technologies both solve and exacerbate environmental problems? Sanjay Poddar: Great question. And there's a duality here between technology and sustainability. You know, when you look at sustainability, and if you look at sustainable development goals that we have, the 17 sustainable development goals, one thing that strikes you that they are exponential in nature.The impact is huge. You know, we are not talking about small things. We are talking about scale. And you cannot do anything at scale without technology. And in this case, if we talk about information communication technology, we talk about artificial intelligence, for example, these are precisely the kind of tools we need today to address the sustainability challenges that we are facing, whether is it climate change, whether it is, you know, issues of building a more inclusive society, for example, biodiversity destruction that is happening. Each of these areas, you need technology, you need AI, you need blockchain, you know, you need digital, right? There is no second thought about it. In fact, we did a survey of companies and we found out that 70 percent of the companies we surveyed, who were able to reduce the carbon emissions in the production, in their operation, they were able to do it because they use artificial intelligence.Now, so there is absolutely no question about the role of technology in sustainability. But what we miss out is, you know, if we are not using this technology in the right way, in the right manner, technology itself has a carbon footprint. Technology can cause a big environmental impact. For example, technology can amplify the issues of bias.For example, privacy. So, we have to make sure that while we use this technology, we have to use it in a very sustainable and responsible way. And the data points, are very interesting. For example, the same AI that is going to help us so much. You know, if you look at AI, you know, you take a large language model like Bloom, which is open source, so some of the data we have, we know.A 160, 176 billion parameter model. When they trained it, you know, I think the carbon emission out of it is somewhere around 24.7 metric tons of CO2 equivalent. And if you look at all its life cycle, including the embodied carbon of the hardware on which it was trained, it goes up to 50 metric tons of CO2 equivalent, for example.And if you take larger models, you know, all the more popular large language models, they may go as high as 500 metric tons of CO2 equivalent. So the same technology that is helping us on one hand is also causing emission, carbon emission. And the impact is not just restricted there, as we know. It is also on other resources like water.You know, we can, you do some, you know, very harmless query to your, you know, the large language models for some questions, "where do I, which other cities I should visit in Turkey in my next trip to Turkey?" Right? You know, you asked 20, 30 questions. Behind the scene, that's half a liter of water that was used.For cooling the data centers, for generation of electricity, and we also know about the other dimension about energy use. So, that's the whole thing. Now, the good part is, we don't necessarily have to have such a severe impact. There are tools and techniques and methods whereby we can design, develop, deploy these systems in a way that they are much more, having lower impact on the environment.For example, they are safeguarding privacy, they give you much more safer response, so you know, there's less bias. So overall, it is very much possible to bring a culture such that the software you write is more sustainable and more responsible. So that's the silver lining, right? So to your first question, a big duality.If you are in business, therefore your strategy, your technology and sustainability strategy needs to be integrated. And you have to look at it very holistically, not just at sustainability by technology. And "how do I use tech to do sustainability," but sustainability in technology, "how do I make sure that the technology is being used in a much more sustainable and responsible way?"Beliz Kudat: Yeah. This is the crucial question as you said, and technology is crucial, as you mentioned in all those sustainability efforts as well. And we also know that software is at the core of all these technologies and companies need to adapt the way software is designed, developed, deployed, as you said, and used to minimize its carbon footprint.So how can they achieve this? Sanjay Poddar: Well, you know, the software stack, there are many decarbonization levers in the software stack. When you talk about a software stack, there's obviously the code itself, which has to be written in a manner that it makes less demand on the underlying hardware, for example, right?So you need to bring that kind of design patterns, architectures, choice of programming languages, all that have a bearing on the emissions or the energy use and emissions. For example, you know, there is a whole study about interpreted languages and compiled languages. You know, a language like C++, if you write a code and you write a similar code for doing the same thing with Python, obviously it is found that the C++ code will need less energy and will emit less carbon.Now, not to say that people have to write in C++ but it's just a data point that, are you even thinking about, you know, which language are you selecting? And then there are, around architectures, for example. And then a very interesting decarbonization lever is the migration of your workloads to hyperscalers, for example, to the cloud.And why does that reduce emission? Because the hyperscalers because of the scale and investments, they invest a lot in renewable energy. They have the right technology, like they use AI, for example, to make sure that their data centers are run with a relatively lower power usage, efficiency, what we call the PUE.So they have the elasticity because of economy of scale. Their utilizations are higher, so the idle time of hardware is less. And now if you see, there is, you know, a lot of investment in what they call the custom silicon chip. And that's the next big thing where you write software with the underlying hardware in mind, optimizing the capabilities of the underlying chips.And now, when you do all this, you know, the code you write, the system you build, it needs less energy. And also because this, cloud centers are typically, you know, you can select where you want to put your workload. You can select a location if your business strategy permits, where the carbon intensity of electricity is lower. In other words, the electricity is more generated by renewable energy, for example. As a result of this, not only you're using less energy, you're also, you know, emitting less carbon. And there are similar decarbonization levers even in the field of AI. You can, you know, you don't need to take the biggest of the large language models.You know, you don't need to use models with billions and trillions of parameters. You have to use the model which is fit for purpose. You have to use the model which gives you the required accuracy. And there are a lot of startups coming up in this field that allow you to do, for example, dynamic routing to a large language model, which has less emission, for example, right?And in the field of AI, a number of different techniques, you can do pruning, quantization. You can write your prompts in a way, you know, so that the overall emissions are lower. It's called green prompting techniques, for example. Probably that's a whole session I can take, but...Beliz Kudat: But I really would like to, I really would like to come to the AI and what can the companies can do about it, especially regarding the energy consumption.But I want to dig in a little bit more in the data centers, because we've been talking about data centers and everybody knows that how they impact the global energy consumption. And, so what innovative technological solutions can be applied here in data centers? And can we specifically discuss softwar-based solutions here?Sanjay Poddar: Yeah, you know, a number of different things can be done when it comes to data centers, and you're right, you know, the data centers are mushrooming, thanks to the generative AI, widespread adoption, in fact, some data points, I was looking, for Ireland, for example, the data center power usage, went, quadrupled from 2015 to 2023 from 5% to 21%.There are cities like London, which is not allowing new housing because there is a challenge of power. The power is being consumed by the data centers. Now, what are the kind of solution one can think about now? First of all, not all data centers are same, right? The data centers, and I also touched upon it in my earlier response, you know, we did a very detailed study for one of the hyperscalers, to understand, you know, if you move a workload from one data center to a hyperscaler, how much emission reduction is possible? You know, anywhere from 50 to 90%, for example. Again, there are several different factors based on which is the hyperscaler, which is the location, and so on and so forth, but typically you will see the PUE of hyperscalers because they run it at scale and for all the reasons I've mentioned, it's far better, right?That is one. Now, from a software-based approach perspective, you know, when you design workloads for a particular, system for to be run on the data center, you can make it much more carbon-aware. Now, what do I mean by carbon-aware? You know, your backup jobs, for example, will run when there is renewable energy, so they are, they're scheduled at the time of the day, or they will be run in a location where there's a bit lower carbon intensity of electricity, right?So, yeah, you know, I'm also the co-founder and chairman of the Green Software Foundation. One of the things that we built was, we have defined is the carbon-aware SDK. So you can, for example, use a carbon-aware SDK to figure out how do you make your systems, you know, run at a time when the carbon emissions are lower, the carbon intensity of electricity is lower.That is one thing. You can build systems which are more cloud native. Serverless architectures, for example. That is the other thing you can do. There are, the software-based solutions that, you know, more advanced data centers use, they use AI, for example, to predict how they can lower the energy that is used for non IT purpose, for example, cooling purpose, right?So they are able to optimize and distribute that energy. So there's a lot of use of AI there. Beliz Kudat: So when you just, I'm sorry I interrupted you, but when you just mentioned the AI, I also want to ask my other question too. Maybe you would like to combine your answers with them because I really would like to, we would like to learn about the tools and methods that can be used to measure the energy consumption of AI and machine learning models, too. So maybe you can... Sanjay Poddar: and, you know, this is, again, an evolving area, but I can tell you what state of the art, because a lot of new things are happening as we speak. But when it comes to AI, you know, there are, you have to look at AI very holistically across its life cycle, right? In traditional AI, people were more worried about training, whereas in the generative AI, people are now more worried about inferencing because that's where more emission is happening.Now, in each of these cases, how do you really measure the emissions happening or energy use, right? So when you are deploying AI, if you are deploying in your own infrastructure, the first thing you can do is the carbon accounting tools that each of the hyperscalers use, give you, right? And then you can use that to figure out, you know, how much emission is happening, how can you lower that?Because you can only reduce what you can measure. So that is, because end of the day, AI is also a workload, right? So you can, that's on the cloud side. And then, you know, there are techniques which are more on the software side that have come up, like the very recent ISO standard by Green Software Foundation called the Software Carbon Intensity Specification, that also can be used.Then there are a host of open source tools, you know, code that can be used with Python libraries. There is a, you know, Cloud Carbon, CCF Cloud Carbon Framework, you know, and again, the Green Software Foundation has created an impact framework. And then I'm also coming across a lot of API calls, which help you open source, which help you to tell how much was the carbon emission for every prompt that you just made, right?And there are a lot of startup systems coming up in this space. So this is a very evolving field. But, it's, coming up with a lot of open source solutions, a lot of solutions from big tech players, from the startup community. That's a big opportunity for the startup community. So that's what you have.A lot of host of tools. The GSF which I chair, we are also currently, focused a lot on the SCI for AI. That's the version that we are working on. Beliz Kudat: Okay, so, of course, there's this issue of ESG goals when we're especially talking about sustainability. So, how can promoting the sustainable use of technology contribute to the companies in achieving their ESG goals, and in attracting talented employees at the same time?Sanjay Poddar: No, I think this is the best question, right? Why should we even do it? You know, I know there's a bigger climate change sustainability thing, so it does appeal to the, you know, all the talented, you know, youngsters who are entering the field, right? You know, but the fact is, they want to work for organizations who are serious about the sustainability issues.So, that's about the talent. So, I'm aware of businesses which are weaving the sustainability messages in their corporate communication because they want to reach to their employees, to the stakeholders about what are they doing about it, and employees want to work for such organizations.There's a lot of research in that area. The other important research, in fact, we did in Accenture, was the correlation between sustainable technology, the ESG score, and business performance. And what we observed in our study was that sustainable technology, organizations which have a strategy on sustainable technology, they have a correlation to better ESG score compared to their peers in the market.And the other interesting fact was that, businesses which have better ESG score, they outperform their competitors 2.6 times in the total, you know, shareholder value they return, right? So, even if you are not as concerned for the planet as you should be, you still have a real tangible reason because your business benefits when you have better ESG score and your ESG score benefits when you embrace sustainable technology.So there's a pure correlation and obviously your employees are asking for it. So I think that is the other big driver for decision makers. These things cannot happen unless it comes from the top. It's a culture change, right? All things that we discussed. And it is a very strategic imperative. And that's what sustainable technology is all about.Beliz Kudat: Sanjay, thank you very much. It was a pleasure having you in our summit. And thank you for all these valuable insights. Sanjay Poddar: Thank you. The pleasure is all mine.Sanjay Podder: Hey, everyone. Thanks for listening. Just a reminder to follow CXO Bytes on Spotify, Apple, YouTube, or wherever you get your podcasts. And please do leave a rating and review if you like what we're doing. It helps other people discover the show. And of course, we want more listeners. To find out more about the Green Software Foundation, please visit greensoftware.foundation. Thanks again, and see you in the next episode. Hosted on Acast. See acast.com/privacy for more information.