BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Podcasts Anne Currie Discusses Cloud Providers and the Environmental Impact of Software

Anne Currie Discusses Cloud Providers and the Environmental Impact of Software

Charles Humble talks to Anne Currie from Container Solutions, exploring the environmental impact of technology. They look at how technology compares to other industries such as aviation and farming, how the big cloud providers compare in terms of their commitments to reducing carbon emissions, and the impact of the choices made by individual developers and software architects.

Key Takeaways

  • Electricity use amounts to about 25% of carbon emissions but is one of the easier problems to solve.
  • Of the big three cloud providers (Amazon, Microsoft and Google), Google and Microsoft are already carbon neutral, and have committed to be carbon zero by 2030.  Google have published a number of papers in which they explain some of the ways they have made their datacentres more efficient to reach these carbon goals.  
  • AWS are lagging behind the other two, and say they will be carbon neutral but not until 2030. They do have some regions that use renewable energy, and where possible it is best to locate your workloads there.  This is a decision that you need to make as early as possible.
  • On the whole, individual code efficiency doesn’t have much impact.  It is much better to use the services your cloud provider gives you as far as possible, as these are likely to be much more efficient.  
  • It is always a good idea to consider whether the work your code is doing is worth the carbon it will send into the atmosphere, particularly when dealing with computationally intensive tasks such as cryptocurrency or machine learning.
     

Transcript

00:01 Charles Humble: Hello, and welcome to the InfoQ podcast. I'm Charles Humble and this week I'm talking to Anne Currie. Anne has been in the tech industry for over 20 years. She's worked on everything from Microsoft back office servers in the 1990s to international online lingerie in the 2000s and to DevOps and the impact of orchestrated containers in the 2010s. She has a strong interest in tech ethics, and indeed helped both InfoQ and QCons work in the space. She championed and ran the first tech ethics track we ever had at QCon London and was the editor for the InfoQ tech ethics eMag which I commissioned in 2018 when I was still InfoQ's editor-in-chief. Away from the tech industry, she's also the author of a science fiction series of books, the Panopticon series, which are fantastic. And she regularly paints in oils, exhibiting in both the UK and the US. Anne, welcome to the InfoQ podcast.

00:53 Anne Currie: Hi.

00:54 What are you currently working on?

00:54 Charles Humble: It's an astonishing list of achievements, and I didn't list all of them because I would've been there all day. So I guess I should start by asking, what are you currently working on?

01:02 Anne Currie: Well, I'm currently on sabbatical from Container Solutions where I normally work and I'm writing the fifth novel in my Panopticon series. And from next week also, I'm the visiting tech ethics lecturer at the University of Hertfordshire. And I do my first lecture next week. They're adding a tech ethics module to their MSC.

01:22 Charles Humble: That's very exciting.

01:23 Anne Currie: Yeah, it's fortunately already a remote MSC. So that's quite easy.

01:26 Charles Humble: Excellent. So what would be the main lesson you want your students to take away from that course?

01:34 Anne Currie: Think. It's think. Think about what you do. Think about all the ramifications and be aware that you won't get all of them. You will make mistakes. There will be bugs. There'll be ethical bugs or legal bugs as well as there are software bugs. And make sure that you can detect them, track them, test, have good reporting systems. Make sure that people can't suffer without being able to let you know and let you fix it.

01:58 Charles Humble: It's exciting to me that universities are teaching ethics as part of the computer science curriculum. I might be wrong, but I didn't think that that was true when I was at university. Admittedly that was a long time ago.

02:12 Anne Currie: Well, I think there always have been some, but I don't know what they used to teach. So this one is not a very academic ethics course, because obviously I'm not an academic, I'm just teaching from what I have experienced through 20 plus years in the tech industry. So I'll start with, what's against the law, don't break the law. It's amazing how much stuff that comes up as unethical was actually illegal. So it might be breaking the Equalities Act, discrimination law. If you don't break the law, then you actually have a good chance of not having done something terribly unethical. So I start with the law and we all know there are loads of laws that you actually run across every day in tech. And you never learn about them at university, you learn about them on the job. But it'd be quite nice just to have a bit of a heads up about what kind of things.

02:57 Anne Currie: The GDPR is the most obvious. Things like contract law, IP law, all of these things are places where you can do wrong without necessarily really thinking about it. That's ethics 101 is the law. 102 is what's going to be the law in future. What can you see by reading the front page of the Guardian? It's not against the law at the moment, but will be against the law in the future, and then try to obey what society is looking like making the law in the future. And the most obvious area there is climate change and sustainable working, which is where I spend a lot of my time these days.

03:31 In terms of carbon emissions how does the tech industry compare to aviation or farming?

03:31 Charles Humble: Right. Yes. And that's really what I want to talk to you about today. This is your second time on the podcast. Last time around Wes interviewed you and he focused mainly on tech ethics and the law and legislation such as GDPR and the like, which you've already mentioned. And so what I wanted to do this time around was to focus on another area that I know is of particular interest to you, which is climate change and the role that technology plays. This was somewhat prompted by a tweet from you when you mentioned you'd been advising Adobe on their climate impact.

04:01 Charles Humble: Now obviously we are in the middle of a global pandemic, but we've also had the Australian Bush fires towards the end of 2019 and the California ones this year, which have both been frankly devastating and have I think made this particular topic top of mind for a lot of people. I feel it's really important and I think it's something that we don't maybe talk about enough within the context of software engineering. So to level set us a bit, could you give us an idea of maybe how the tech industry compares to say aviation or farming?

04:31 Anne Currie: There are good sides and bad sides to the tech industry compared to other industries. So we have two impacts really, one is hosting. The biggest kind of single impact is hosting where all our servers in cloud, in your private data centers, in colos across the world is about 2% of the world's electricity use. It's not exactly the same as, but it's the order of magnitude the same as the aviation industry uses in terms of energy. So obviously the aviation industry do get a lot of stick, we don't because generally the public aren't that familiar with the concept of data centers. So we get away with it and we haven't had much pressure up until now, although I think people are becoming more aware of it now, and we are coming under more pressure.

05:12 Anne Currie: The other impact which is greater actually is charging all your phones, charging all your laptops, everybody everywhere, charging all their devices and producing devices. But I'm less worried about that because that's an awful lot of individual effort and you know, however you personally get your electricity and all that kind of stuff. But what we can focus on, which is entirely within our control is server hosting. And there has been a very interesting move in the past couple of years on that. We have got better, but we need to get better faster, and possibly expand what we're doing out more widely.

05:46 In 2018, you helped launch the sustainable servers campaign. What was the trigger for that?

05:46 Charles Humble: So let's unpack that a little bit more. In 2018, you helped launch the sustainable servers campaign. What was the trigger for that? What prompted you to launch that particular campaign?

05:57 Anne Currie: Oddly enough, it came out of the tech ethics track that we did, and then I ran another conference after that. What should we be doing? What's the low hanging fruit? Where can we have a big impact in the tech industry? Something we're doing badly, but we can do better. And a couple of others, including Paul Johnston, who we've also had on InfoQ before and QCon before. We decided that the best thing that we could possibly do in the tech industry was just say, look, we're going to run all servers on sustainable power. So, carbon zero. How quickly can we get the entire tech industry to carbon zero? It's a very achievable goal. You mentioned before about agriculture.

06:37 Anne Currie: I don't know if anybody, I'm sure some of the listeners here will read the Bill Gates newsletters, which are very good, very good on climate change. Something that he points out is that electricity is the low hanging fruit. It's about 25% of carbon emissions. And really there's no excuse for not sorting that out because electricity is completely replaceable with solar power, with wind power, and it's getting cheaper. There's no reason why everyone who uses primarily electrical power, can't move over to carbon zero extremely quickly, and we should be doing that. So we in the tech industry are lucky because we're talking about electricity. We have it easy. It's much more difficult with agriculture and construction and industry, and they make up 75%. And actually, if you look at the Bill Gates notes these days, he's mostly focused on how can we do something about that because that's a much more difficult problem. From where we are in the tech industry now, we should be able to sort out our contribution very easily, if we try.

07:38 Charles Humble: And we have seen considerable progress on that in the last couple of years, right?

07:43 Anne Currie: Coincidentally, probably, since we started doing the sustainable servers campaign, we've seen all of the cloud providers really step up to the plate on that. And they now all have compelling commitments on achieving carbon zero across their businesses. So at the moment, my focus is just to try to keep the pressure on so that they maybe move those dates forward, but we certainly don't let them move back.

08:08 How are the big three cloud providers doing in terms of actually getting to carbon zero?

08:08 Charles Humble: So commitments and targets are obviously welcome, and it's great to see that happening, but when we think of the big three cloud providers, so, Amazon, Microsoft and Google, how are they doing in terms of executing? How are they doing in terms of actually getting to carbon zero?

08:23 Anne Currie: Oddly enough, Microsoft kind of set the stage for this last year. And Google had been doing it for a long time, but they followed suit. Although Google do things, they don't tend to talk about them as Microsoft, they'll do things a bit later, but they'll really make a song and dance about it. And then Amazon followed after that a little bit. They haven't gone all the way, but they have gone some way. And the path appears to be one, carbon neutral. And carbon neutral means, okay, I'm still going to put carbon into the atmosphere, but I'm going to offset it. I'm going to buy credits or I'm going to pay for trees to be planted, or I'm going to pay for new sustainable power stations to be built somewhere else. So somebody else isn't going to put the carbon in so I'm offsetting my carbon. So it's good. It's step one.

09:08 Anne Currie: And there are two positive things about carbon neutral. One, you're actually tracking how much carbon you're producing, which is incredibly useful for you. And two, it's putting money into the business of sequestration or the sustainable power business. Now, if the world was not expanding, if there was no growth and we weren't under particular time pressure, then you could just use carbon neutral and just let things gradually, gradually, gradually ease over to being entirely sustainable that way because a lot of people putting money into new wind farms and new solar farms, that would be all that anyone was required to commit to.

09:43 Charles Humble: And Google are already carbon neutral I believe, although there seems to be some debate about exactly when that happened.

09:49 Anne Currie: Sometimes they say 2007. Sometimes they say 2017. There tend to be little bits about whether they were buying it on credits or whether they were doing it in other ways, whatever. But it basically, let's say 2007 I think is actually the comparison. Microsoft has been doing it since 2013.

10:07 Charles Humble: And have Amazon committed to doing it as well?

10:10 Anne Currie: Amazon have not yet committed to do it. Well, they've committed to do it, but they will not have it done that until 2030 I think? Which is terrible. They are a decade, more than a decade behind their competitors, which is quite astonishing. But they do have four sustainable regions where they are carbon neutral, which are Ireland, Frankfurt, Canada, and Oregon. And if you host there, they will buy credits or whatever they do. I think they buy credits to make sure that whatever you're doing is carbon neutral. So you can at least feel that they're doing it for you. It's the right thing to do. If you're going to host on Amazon, you should host in one of those regions.

10:46 What does carbon zero mean?

10:46 Charles Humble: So that's carbon neutral. And then beyond that, we get to carbon zero. So what's the idea there? What does carbon zero mean?

10:53 Anne Currie: The idea of carbon zero, you'll hear this talks about a lot, particularly after China made a statement this week that by 2060 China will be carbon. Actually they said carbon neutral, but I think they actually meant carbon zero, but no more carbon released into the atmosphere from anything that you're doing. So Microsoft and Google have committed to that for 2030. And how that normally works is either you power your data centers directly from a wind farm or a solar farm or whatever, or it's actually in Oregon and in Canada. Hydro is a very good way of doing it. Or you get it from the grid, but you make sure that it's time matched. So you only pull electricity off the grid at a time when you know that there is enough renewable energy going into the grid. So that usually requires a lot of battery use to smooth out your workloads so that they tend to be focused on when there is electricity to run them.

11:52 Anne Currie: Now, I don't know if many of your listeners have come across, Google did an awful lot of early work in data center efficiency. They may have read the Borg paper, for example, about moving workloads around so that you didn't run workloads that were not actually latency sensitive at times when it was expensive to do it because there were lots of latency sensitive workloads that needed to be run at that time. And that way you smooth out your load, so I'm guessing they're doing much the same stuff for this. They're trying to work out and prioritize workloads so that they run at points where there's electricity and don't run at points where you need to save the electricity for latency sensitive stuff.

12:31 Charles Humble: There's also the work that DeepMind was doing that they publicized in 2016, where they applied their AI to Google's data center cooling bill and reduced that by 40%. DeepMind of course was acquired by Google in 2014 so there's another bit of Google and they have been doing really interesting work in this space.

12:46 Anne Currie: So they have done well, but we do need to keep the pressure on because it's so important, particularly on Amazon, because they only really started talking about work on this but I suspect work on this last year really. Amazon do things because their customers demand that they do. And we've started to demand this now and so they're acting. But we didn't before. So in some ways, Amazon are the baddies. They're the ones that we have to keep the pressure on.

13:12 Anne Currie: But I also consider Amazon to be the last best hope for peace in this, in that Amazon are astonishingly good at changing how businesses work and how consumers work. So what they do once they've done this and they really seem to be investing in it and they really actually seem to be investing directly in the production of power. It's my dream that one day we would get power from Amazon that was all renewable because that's what people were demanding. And I think they could do it. They're one of the few companies in the world that I actually think could change how we do things very quickly. They're surprisingly competent. I want them on my side.

13:53 I presume if you're on AWS, then your choice of region makes a substantial difference to the environmental impact that you're having?

13:53 Charles Humble: Absolutely. As you say, Amazon does have a track record of being able to make these kind of very substantive changes when it chooses to, and hopefully that'll happen in the context of environmental impact. I certainly hope so. Their track record to date isn't great. There's the report that Greenpeace published last year, it came out in February of 2019 where they said that Amazon is only meeting 12% of its renewable energy commitment, specifically at its East Coast presence. And this I think is a general problem with data centers in Virginia, which I mean, Virginia is a kind of coal-fired state, I guess. And that's unfortunate because there are a lot of data centers in Virginia. So I presume if you're on AWS, then your choice of region makes a substantial difference to the environmental impact that you're having?

14:44 Anne Currie: Absolutely. It's all about location. There are some locations where it's really easy to generate renewable power and some locations where it is incredibly difficult or there's too much demand for that to be easily done. And one of the places where it's incredibly difficult is Virginia. So everybody built their data centers in Virginia because it's where the transatlantic cables came in. So there's an awful lot of electricity demand in Virginia, but in a horrible perfect storm, the state of Virginia is very uninterested in renewable power. They are a coal-fired state. And so that electricity that's running on those data centers is dirty and it will be very difficult for them to convert Virginia to produce enough clean energy in Virginia, just because there's so much demand. So it's the worst place really. I would strongly recommend that people try and find a way not to have their instances in the East Coast. I mean, I know it's always been the default, but it's the hardest place probably in the world. I don't know. I'm sure there are worse places, but there probably aren't to do your renewable power.

15:49 Charles Humble: It's difficult to shift once you're up and running and it sort of scales. So that's really a decision you kind of need to be making upfront. If you hadn't already moved and ended up in EC 1, then that's the time that you can really do this, but otherwise you're kind of stuck, right?

16:05 Anne Currie: Yeah. This is a decision you need to make as early as possible. It's something that you need to be aware of because I don't think any of us think that there's a problem with being locked into your cloud provider because that's fine. Cloud providers do things far, far better than you can. Generally, it's a risk lowering activity. But you're not just locked into your cloud provider, you're locked into the cloud providers regions, and that does make much more of a difference than you think. So you do need to think about it upfront.

16:30 Anne Currie: And I would really love it from Amazon if they would give us more of a heads-up on what regions we should be. I mean, we've got the four, but people are always saying to me, "Well, I'm in Australia," or "I'm in Asia. Where am I supposed to put my instances?" And I can't say, I don't know. I don't know what's better. So I don't know which regions around there are easier for Amazon to power renewably because they play their cards very close to their chest. They're not very transparent and they don't tell people. And I would really love them to tell us more stuff so that we can help.

17:06 Charles Humble: And that's in something of a contrast to both Google and Microsoft who have been quite open in terms of what they're doing and the efforts that they're making, which I find that just interesting.

17:13 Anne Currie: Yeah, it is. And to be honest, I mean, I don't know. I don't know why Amazon has been so closed mouth about it. Probably because their story wasn't very good, I would guess. But maybe that's not the, maybe they just like...

17:25 Charles Humble: Yeah, I would say it's a reasonable assumption, doesn't it?

17:28 Anne Currie: But it might just be that they didn't want to tell anybody what they were doing. I mean, they don't like to say an awful lot about what they're up to. And quite often it's because maybe it was a little bit belt and braces to start with and they didn't necessarily want folk to know that. Certainly things like Lambda, which should be fantastically good for efficiency and lower energy use, in the early days it sounds like it was really very inefficient and these days it's very efficient, but we were certainly all talking about its potential efficiency back in the days when we were under the impression that it was efficient, but it really wasn't.

18:04 You mentioned that you've been advising Adobe recently on their climate impact so can you talk a little bit about that. What sort of things did you discuss with them?

18:04 Charles Humble: Not at all. Yes. Now you mentioned that you've been advising Adobe recently on their climate impact so can you talk a little bit about that. What sort of things did you discuss with them?

18:11 Anne Currie: Advise probably is overstating the case. They asked me to come in and have a chat to their lunchtime brown bag about sustainable hosting. It was very interesting. Lots of what we talked about was what we're talking about now, which is it's all about location, location, location. This didn't come across a secret, but they use Amazon and they use other cloud providers as well and we talked a little bit about which regions are the best ones and what you should be thinking about. Something that came up a lot in the conversation I had with them, which was something that comes up a lot generally whenever I speak about this, and I have a lot of sympathy with this because I used to think it myself was the people immediately say, "Shouldn't I just be making my own code more efficient?"

18:50 Anne Currie: Before I got into this I co-founded a startup around data center efficiency and I'm a big believer in it, but I changed my mind about whether it's a good thing for end users, us, to do because the cloud providers just do it so much better than we possibly can. And they're so very good at dividing up machines, either using containers or using VMs so that if you've got an application that's inefficiently using, it's using a lot of CPU but it's not using a lot of memory or whatever, they will just divide up the machines to get excellent utilization across the whole data center or across multiple data centers in a way that you just can't. And so it's an awful lot of effort for you to be constantly optimizing, and it's not what your business is for, whereas it is what their businesses for. I'm tempted to just leave them to it.

19:44 Charles Humble: So presumably then the advice would be, use as much of the stuff that your client provider makes available as you can, and only code the business logic and the bits you absolutely have to code and use the cloud provider stuff as much as possible, because that will be more efficient. Is that right?

20:01 Anne Currie: Yes. Well, it was a bit like Lambda. Some of it might not be efficient now, but in the long run it will be vastly more efficient. So I would say that's the bet. Place your bets with cloud providers, place your bets using the cloud in everything that you can is, I think people call it the cloud native approach, but yes, I do think that that is the way to go.

20:20 Can individual developers make a difference?

20:20 Charles Humble: So given that what we're saying is as an individual software developer or as a software architect, the best thing that you can do if you're running your workload on the cloud is to take advantage of all the facilities that that cloud provider gives you because that's the most efficient code and therefore that will minimize your environmental impact. Can individual developers make a difference?

20:41 Anne Currie: Oddly enough, they do. There's a war going on between the cloud providers to look the most green and some of them even to be the most green. I think at the moment with Amazon, it's hard to tell whether they're looking green or they are green, but they always look green before they are green. So let's say I'm happy at the moment there to be this war. And the war is because they think developers care about it now, which they didn't previously.

21:05 Anne Currie: I said this at Adobe, and I say this generally, I know you all want to do something crazily difficult, like re-architect your entire product, but actually the win here is probably something which is trivial, which is just to say to your cloud provider that you care about this. Tweet about it, blog about it. You don't even need to do that. Just say to the rep who you talk to, "I care about this. I want you to tell me where I should be hosting to be most sustainable and I want you to be more transparent about what your plans are in the future." This is a win that will come by exerting consumer power, not developer power. But developers are consumers. They are the only consumers of the cloud providers. Use your consumer power.

21:51 What is your view on situations where you're running very computationally expensive workloads?

21:51 Charles Humble: And thinking about individual developer responsibility more broadly, what's your view on situations where you're running very computationally expensive workloads? I'm thinking about things like cryptocurrency or maybe machine learning in some cases.

22:07 Anne Currie: Obviously, I really dislike computationally expensive. Some of them are hosting in regions where there is a lot of cheap, renewable power. Iceland is a good example, but they're having to put in new hydro stations in Iceland specifically for cryptocurrency mining because it's so common. It's not appropriate. It's completely inappropriate to be doing something like that at a time when we're in a climate emergency. It's just the wrong thing to be doing. Now, whether I think it's a cool technology or whatever, doesn't matter. It's the wrong thing to be doing right now. They need to be moving as quickly as possible to proof of stake, not proof of work, because it can be done. You don't have to do it through proof of work. Just put the effort in, move it over to proof of stake. In the meantime, it's completely unacceptable.

23:01 Anne Currie: Now, an interesting question is machine learning because machine learning there is purpose to it. I mean, sometimes there's totally no purpose to it whatsoever and it's nonsense. And that's one of the things that as a developer you should be thinking about, is this worth the carbon I'm about to put into the atmosphere? Should I hold off until I can actually run it on a carbon zero server? But also it's really well-worth developers thinking about the fact that machine learning, like cryptocurrency mining, is not latency sensitive. It doesn't have to run in your nearest data center, which might be in Virginia. It can run in a data center on the other side of the world where there's a lot more access to renewable power or it can run at times of the day when there's a lot of renewable power and not run at times of the day when there's not.

23:52 Anne Currie: So it's bad, but you can manage it if you think about it. It's a very good example of where I suspect you'll get an awful lot of benefit from doing it using a cloud provider service to run your machine learning for you. I mean, if you run it on Google, I would guess, and this is a complete guess on my part, but I'm 99.9% sure my guess is correct that they will look at your machine learning load and move it to a time or a location where it can be run reasonably neutrally or carbon zero. They can because they've got so much load and they have so much of an understanding of their systems and they care.

24:32 How are the other big tech companies, Facebook, Apple and the like,doing in terms of climate impact?

24:32 Charles Humble: We've talked a lot about the sort of well-known cloud providers and we've talked also briefly about Adobe. How are the other sort of big tech companies, I'm thinking about Facebook and Apple and the like, how are they doing in terms of climate impact?

24:44 Anne Currie: Well actually, some of them are very good. In fact, Apple and Facebook are particularly good. Facebook does an awful lot of work to keep its data centers efficient and renewably powered. They're all carbon neutral now. I don't know about their carbon zero targets, but I imagine that they will have some. Again, very good. Anything that you are not running yourself that you're handing over to somebody else and they're completely owning, they have the chance to do good work and fortunately, those folk all are doing good work. Generally Facebook are appalling, we hate them, but I do have to occasionally say, but they are quite good at climate change.

25:19 Is there anything specific you would like to see from the big cloud providers that we haven't seen?

25:19 Charles Humble: We are coming up towards the end of our time. I was wondering if there was anything specific you would like to see from the big cloud providers that we haven't seen?

25:28 Anne Currie: Talking. I would like to see them exerting their power outside of tech. Google are so quiet about these things. I would love to see them talk more, but I think that's an unrealistic expectation. Microsoft are very good at not only saying, "We are green, but we want everybody to be green who is a Microsoft user or a Microsoft supplier." They're going out, there talking about what they're doing. They're talking about how they ran their early carbon internal credit tracking systems and things like that. I would like to see them do even more of that. I think that's fantastically good. And I'd like to see folk like InfoQ talk about that because then everyone can know what good work they've done there.

26:05 Anne Currie: And I would really like to see Amazon do more and talk more about what they're doing. I've said this before, and I will say it again, my dream is that Amazon make power their next big business area. They have a tendency to do that. AWS was because they used it internally and then they made it available to everybody else and it kicked off the whole cloud revolution really. They're going to have to build an awful lot of renewable power systems themselves, batteries, wind farms, solar farms, everywhere to power AWS. Once they've done that, they'll have built a hell a lot of expertise in doing that. I would love to see that be the next AWS. That would be my dream. I've no idea whether that's happened. That's absolutely based on no evidence whatsoever except a wild hope.

26:55 Charles Humble: That's wonderful. Anne, thank you very much indeed for joining us on the InfoQ podcast this week.

27:00 Anne Currie: Thank you.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article

Adoption
Style

BT