Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Mike Long on Modern C++ and the C++ Memory Model

Mike Long on Modern C++ and the C++ Memory Model


1. We are here at Buildstuff in Lithuania, I am sitting here with Mike Long. So Mike, who are you?

I am a software coach and I’ve helped teams adopt modern software development practices in hostile legacy embedded systems.


2. You gave a talk here on something called the C++ memory model, what in the world is that?

So the C++ memory model is a new addition to the C++ standard, it came in 2011 and for the first time the C++ standard defined semantics for how multi threaded code should behave, for instance how reads and writes become available to other threads and how reordering can happen.


3. Maybe you can explain some more, you say “when a write becomes available to another thread”. But when I write to memory, it is available right?

Well yes, in the programmer model you have your editor and you write your code and the code executes in a linear fashion, as you’ve written in program order. And that’s the natural way we think about things. But actually that’s not what happens when we run our code, the compiler reorders everything, it has all kinds of optimizations that move the code around to make better use of the hardware, and the hardware itself makes a lot of changes too, for instance it does out of order execution, or pipelining or speculative branching. And all these things conspire against you and your program order.


4. It’s mainly the compiler who throws a wrench in the works right, that reorders things?

The compiler is a big part of what was missing, obviously this was a change to the compiler standard, the language standard which you think about in terms of the language. But actually what the standard defines is a contract between you as a programmer and your execution environment and in C++ it’s defined in terms of an abstract machine, not like this is how your C++ code will run on x86; what it says is this is how the code should run on an abstract machine and then it’s up to GCC and Intel or Microsoft to make all that happen. And not just on the compiler level, but as a contract between the compiler and the hardware and the programmer. Because the hardware is becoming more and more complex now we have lots of threading happening on the hardware. So all this reordering stuff is not enough to define it at the compiler level.


5. When you say the hardware reorders operations like out of order, isn’t the CPU taking care of making sure that everything looks sequential or does that leak through?

It’s the job of the compiler to – if you like, in C++ we have this rule we call the “As if rule” so the compiler can do whatever it wants, and the hardware itself can do whatever it wants as long as it looks as if it had run the code that you wrote. So it can reorder things any each way it wants as long as at the end of the day you can’t tell that it changed the order. And that’s fine if all of that happens in a single thread, but all those optimizations in a multi threaded context can have serious consequences, for instance if you have a boolean flag to say that a value has been initialized, after initialization, if the compiler decides to reorder that and writes the flag to “true” before the variable is initialized then in another thread you are going to see a different view of the world, so with that kind of magic the illusion is broken. The “As if rule” works and is defined on a single threaded basis, so the memory model what it does is it adds the semantics for how these things happen between threads.


6. Can you give us a more specific idea of what that looks like? Is that something that the developer sees or is it just a guarantee between the compiler and the machine?

So the memory model is defined in many ways but usually what the programmer will see is the use of atomic variables or use of threading primitives so that’s the code level if you like, and underneath that code level, the standard will specify the semantics for how these things will behave in a multi threaded context. So for instance if you have an atomic int and you write to it, then what the compiler say is “Ok, we’ve written to an atomic end so everything that happened before you in program order must have already happened, I can't reorder that stuff past here” and all the stuff that happens after this, must happen after also. So, it forms a memory barrier, this is an old fashioned memory barrier for kernel programmers and what it says is “Ok, stop the world, everything has to have happened and all the stuff that is still to happen must come later”. So if you are looking at another thread waiting for that write to appear, then it all makes sense, you're guaranteed that when that change happens all the other preceding operations have occurred also.


7. Does this have an impact on performance because you are effectively restricting the compiler?

Yes, these memory barriers are expensive, and that’s one type, the memory barrier I just talked about is called sequential consistency, and this is the default option in C++ but it’s also the most intuitive model and it’s the model that for instance Java uses in its volatile keyword, there are other types of memory ordering available in C++ and they are called weak atomics, and these weaker atomics give more freedom to the compiler to reorder things, for instance the release and acquire semantics where they say “Ok the release is only happening for the write and so anything that happens beyond that can still be reordered past me” like bring it into the future, but all the stuff that happens before me must have happened so you can give the compiler a bit more freedom by using weak atomics but they are quite difficult to reason about, unless you really think it through. But there are certain cases of applications where these weaker atomics are really important, because you don’t want to have these hard memory barriers when you don’t need them.


8. These memory barriers do they have to do with cache coherence protocols? Or is that on a different level?

So, cache is like the memory subsystem is another part of the equation where the world conspires against you. What is a write? Is it when it gets into the register or when it’s written to cache, or when it gets flushed out to main memory. And there are costs associated with each of these things and if you have multiple processors, what does it mean to actually acquire a variable from another thread. These are all things that need to be defined.


9. How are these things, like a memory barrier, how is it actually implemented on the machine? Is it an instruction?

That’s an interesting thing, this stuff has always been available in the hardware, at a low level. So some compilers will give you intrinsics or you might have to do some assembler to get down to that level but on the hardware these things have always existed at least for as long as there have been these options. But weak atomics and these weaker memory barriers are there in your instruction set and people are using them. So what C++ did they just gave a kind of a higher-level abstraction that was portable as well across different platforms and had specific semantics. So there is nothing really new, it’s just that C++ is given these low level primitives and made them available in a clean way.

Werner: It has finally cleaned up its act basically.

I am sorry?

Werner: It’s basically cleaned up its act.

Well, yes.


10. Why is this happening now, why not ten years ago, twenty years ago?

Well I mean the short answer is the free lunch right? Everyone says the free lunch is over C++ has always been a language of choice for performance critical systems. But what performance means now is different from what performance meant twenty years ago when C++ was defined. Now we are starting to get into a world where we are not getting more processor cycles, but we are getting more cores. So that means we are naturally moving towards, to exploit those cores we need multi threading, so the language needs to move into this area, it needs to be able to handle this multi threaded world. That’s really the driver. Where the memory model came from? It was actually from the land of Java, Java also realized after the language had been defined that it needed a memory model, it needed to define semantics for multi threaded programming. And it did that, so a lot of the learning from the Java working group went into the C++ memory model.

Werner: You brought up the C++ language so I think we are now at C++ 14.

Yes, that is coming.


11. Is it still the same language that Bjarne created 30 years ago or has it changed a little bit?

Most definitely not, I mean the language has evolved a lot since its inception but even in the last five years it has changed a huge amount. The last big standard was in 2003 but then it was 2011 before we got the next standard and there was really a kitchen sink kind of change, like everything changed, we got lambdas, we got threading, we got a memory model, lots and lots of new features. And so the way to write modern C++ code is very different from the way for instance I learnt to write C++ code, you need a different approach. And learning all these new techniques is important.


12. [...] How do you as a programmer decide what to use, what features to use? How to structure your application?

Werner's full question: Looking at the C++ language, it's a huge standard, and let’s imagine, I am not a C++ programmer, but if I came into it how would I find out what to use, I mean, namespaces, lambdas, functions, classes, generics, it’s a whole different topic. How do you as a programmer decide what to use, what features to use? How to structure your application?

It very much depends on your application as well because C++ is a language where you only pay for what you use, so for instance if you don’t want to use exceptions for example you can have your runtime without exceptions and it would be fine. If you want to trim it down, not use standard library, not use memory allocation, have it in embedded devices, you can do that. But on the other hand if you want to write cross platform multi threaded computer games, that’s ok too and the subsets of the language you take would be different depending on your application of the language. So whether you need generics or whether you need threads or whether you need exceptions, you would pick your subset of the language but it will depend very much on your application.


13. So maybe a question from a different perspective is which language features would you not touch with a ten foot pole?

Actually I might say the C++ memory model, when you are learning anything, you should go with the simplest thing you want to have the smaller subset that you can be effective with and not create a mess. And C++ is a language that gives you an awful lot of freedom to shoot yourself in the foot. It’s always made that compromise it’s gone for giving programmers power over giving them safety. So with that in mind you probably want to stick to the safer aspects of the language until you’re comfortable with these things. So for instance when you are thinking about the memory model, and what kind of atomics you want, I would go for sequential consistency, they are obviously the most expensive kind of atomics but unless you really know what you are doing, they are the safest kind to use so you should pick those.


14. Stay away, don’t fiddle with the memory model, is essentially a good advice, unless you really know what you are doing

Exactly I mean weak atomics are for a very special case.

Werner: But of course they are exciting because they are new and they are tricky.

Yes, these are great things to play with, and to learn from, I mean you learn a lot about hardware architecture, memory barriers, how a lot of important systems work. For instance the Linux kernel they have a great how to on memory barriers, it’s in C but all the principles are valid and it’s really interesting to look at how an operating system kernel how it needs to think about memory barriers in order to do symmetric multi processing.


15. One other question is the addition of lambdas, how has that influenced the design of code or how has it influenced you?

ally not so much I mean it’s become nicer to write functions that can be passed around and for me it’s great there’s a lot of good syntactical sugar, we’ve always had this function objects that we could create, but the amount of code you had to write to create one it was just painful. So it’s a nice abstraction and makes things simpler, and it pushes people more in the right direction, let’s have lots of small little functions, let’s go more in the Ruby kind of style where we are going to do a map or we are going to do a filter, think about these operations in a more functional way. Yes there is definitely more functional thinking going on but I don’t think it’s the biggest change that has happened in C++ although it was one of the more difficult aspects to integrate into the language because adding new syntax is tricky to an old language.

Werner: Particularly C++ which is impossible to parse.

There is a lot of syntax in C++.


16. So with languages the obvious question is always what would you like to see in C++ 2020, is there anything you want or is the current language fine?

I wouldn’t say I want anything in particular but what I would like to see and this is the way the community is moving towards, is more frequent changes to the standard. And having the compilers evolve along with the standard. So for instance when the C++ 14 standard is stamped, there’s already compilers that adhere to the standard, and that’s great like with C++ 03 it was a long long time before there was adoption in the compilers. So now the compiler vendors and the standard body are working together and evolving the language in real time together and having more frequent releases.

Werner: That’s a big change I think recently because it used to be, a new C++ version came out and you had to wait ten years before you could use it.

Yes, this is all great because we are not only going to get a new standard, but we are actually able to use it straight away. So as a programmer this is fun, we’ve always got new toys to play with.


17. To wrap up, what areas would you use C++ nowadays, or what are areas where you say don’t use C++?

For sure there are lots of areas where you should avoid it, I mean I wouldn’t write any web applications in C++ but there are a lot of very good places – I mean if you look at all the places that C++ is used, it’s used for a reason: it’s either embedded systems or a resource constrained systems with hard real time problems, it’s computer games and graphics where you need also real time performance, so getting the most out of the hardware is critical, and it’s also big super computer scientific applications where you are trying to crunch huge amounts of data. In my experience all these are great places to use C++. Where it’s not so good is obviously where a software language that can help you be more productive would be nice, there’s a lot of business applications where I wouldn’t consider C++ to be a good solution.


18. Final question, if I were to take away your C++ privileges, what language would you move to?

That’s pretty tricky, I would either fall back into C because I am actually very fond of C because it’s such a simple language, so I would probably go for C and if not C then I would go at the other end of the spectrum to either something very functional like Clojure, or something very flexible like Ruby.


19. What about Rust?

Rust is interesting but for me I like to be on the mainstream when it comes to language simply because the ecosystem is so much stronger you get much more help, the knowledge base is greater, there’s more open source, there’s a lot more interoperability, so these languages are great and I am very interested in the way Go is tackling problems, but I am probably not ready yet to – I wouldn’t do a production system in them, I like to play but for sure not for something commercial.

Werner: Thank you very much Mike.

Thank you.

Jan 28, 2015