An interesting discussion about the merits of native code versus Just-In-Time based systems has recently taken place with Herb Sutter of Microsoft and Miguel de Icaza of Mono both providing insightful commentary. Taken together they have provided an informative look at the current state-of-the-art in between native and managed code.
Herb Sutter began by answering the question, "When will better JITs save managed code?" Sutter summarizes his position with the statement that "C++ and managed languages make different fundamental tradeoffs that opt for either performance or productivity when they are in tension."
Sutter views C++ as settling the tradeoff by opting for performance whereas managed languages (which he defines as Java/NET) opt for programmer productivity. Providing more depth, Sutter goes on to say:
"First, JIT compilation isn’t the main issue. The root cause is much more fundamental: Managed languages made deliberate design tradeoffs to optimize for programmer productivity even when that was fundamentally in tension with, and at the expense of, performance efficiency."
Miguel de Icaza picked up on Sutter's points and provided his perspective shortly thereafter.
“Designers of managed languages have chosen the path of safety over performance for their designs. For example, accessing elements outside the boundaries of an array is an invalid operation that terminates program execution, as opposed to crashing or creating an exploitable security hole.”
The two differ on how each of them view the ability of a JIT based language to generate optimal code:
"Second, even if JIT were the only big issue, a JIT can never be as good as a regular optimizing compiler because a JIT compiler is in the business of being fast, not in the business of generating optimal code." – Herb Sutter
But I have an issue with [the above]. In general the statement is correct when it comes to early Just-in-Time compilers and perhaps reflects Microsoft's .NET JIT compiler, but this does not apply to state of the art JIT compilers. – Miguel de Icaza
de Icaza feels that JIT compilers must strike “...a balance between the quality of the generated code and the time it takes to generate the code. JIT compilers tend to go for fast compile times over [the] quality of the generated code.” However, Mono allows users to choose higher quality in their generated code by using LLVM to compile. de Icaza notes that “[Mono uses] the same compiler that Apple now uses on Lion and as LLVM improves, Mono's generated code improves.”
The ability to select the LLVM backend allows Mono users to choose between compilation speed and the quality of the generated code according to the needs of their particular project. This provides greater flexibility when developing as frequent recompilations are necessary during development and reducing compile time speeds development. Then when a project is at or near completion the selection for optimized code can be made to benefit end-users.
This is not to say that the choice of safety over performance does not mean managed runtimes cannot improve their performance further. de Icaza offers some areas for improvement, including intents, extending VMs, limiting dynamic language features, and better support for intrinsics that would map language features on to native hardware.
Sutter posted a follow up comment to his original post, and feels there is a fundamental performance advantage to C++:
“But there’s always an inescapable and fundamental difference between “prevention” and “cure” — when it comes to performance optimization, C++ always chooses “prevention,” and managed languages choose “cure” with the above-mentioned heroic efforts and many more. But the old ounce/pound saying is inescapable; you can’t beat prevention (in part because you can always add the cure after first doing the prevention, but not the reverse), and if you care about performance and control primarily then you should use a language that is designed to prioritize that up front, that’s all.”
How do InfoQ reader's feel about the tradeoffs between C++ and managed code? Do the developer advantages managed code promotes make up for any performance shortfalls? Does modern C++'s new features require a second look from managed code developers?
Community comments
Prevention vs. Cure
by Jerry Lam,
Re: Prevention vs. Cure
by Serge Bureau,
State of art JIT produces more optimal code
by peter lin,
Re: State of art JIT produces more optimal code
by Alex Hornby,
Managed code is more than JIT alone
by M Vleth,
Welcome to 1995
by Cameron Purdy,
Re: Welcome to 1995
by Paulo Pinto,
Re: Welcome to 1995
by peter lin,
Re: Welcome to 1995
by Charlie Helin,
Both are missing the point.
by Paulo Pinto,
You can put lipstick on a pig
by Dan Tines,
Re: You can put lipstick on a pig
by peter lin,
Re: You can put lipstick on a pig
by Paulo Pinto,
Re: You can put lipstick on a pig
by Dan Tines,
Test and Change
by Glenn Graham,
Space vs Time
by Russell Leggett,
Re: Space vs Time
by peter lin,
Re: Space vs Time
by Charlie Helin,
Prevention vs. Cure
by Jerry Lam,
Your message is awaiting moderation. Thank you for participating in the discussion.
I really like the analogy about prevention and cure. C++ and Java are both languages I like to use. It would be nice in the future that C++ also provides features for cure (things that C++ programmers don't want to mess with) and Java can provides some features for prevention (things that Java programmers want to have more control of for example false sharing).
State of art JIT produces more optimal code
by peter lin,
Your message is awaiting moderation. Thank you for participating in the discussion.
Anyone that has followed Azul and Cliff Click knows that JIT is better at optimizing code than static compiler optimizations. For a small program, the developer can flip compiler settings, but not for a large application. Once the codebase is over 100K lines of code, developers have a hard time figuring out where to optimize. Usually, you have to profile the code under a variety of loads to figure out where to tune. Apps with 1 million lines of code become practically impossible to tune. It can be done, but you have to spend a lot of time and effort. Even then, it requires a level of skill most C++ programmers do not have.
Re: Prevention vs. Cure
by Serge Bureau,
Your message is awaiting moderation. Thank you for participating in the discussion.
It is interesting to see experts that can forget crucial aspects.
A JIT has dynamic informations not available to a compiler.
So in fact as JIT improves they should offer better performance than any compiler`s.
There is no amount of static analysis that can match the data from real runs.
So at first runs, compilers have an advantage but on programs running for a while JIT should be favored. And as evidence of this, many Java programs are faster than C++ equivalent.
They both missed that fact.
Re: State of art JIT produces more optimal code
by Alex Hornby,
Your message is awaiting moderation. Thank you for participating in the discussion.
Agreed. Also the JITs runtime analysis can allow some optimizations static compilation can't do, such as inlining of virtual method calls. This can make a big difference on real world code bases.
Managed code is more than JIT alone
by M Vleth,
Your message is awaiting moderation. Thank you for participating in the discussion.
Let's go even further with the potential. A managed runtime can in fact determine that dynamically assembled call stacks are readonly and don't use any IO and can therefor be cached. Or determine that thread synchronization is not necessary even though it's in place in the code. There are probably a lot more of those kind of optimizations possible with a managed runtime.
Welcome to 1995
by Cameron Purdy,
Your message is awaiting moderation. Thank you for participating in the discussion.
Herb Sutter is stuck in 1995, and he's also wrong. The C++ compiler only knows what can be known at compile time, and thus it is forced to produce inefficient code, because it doesn't know whether the data structures are contended, whether the code is single-threaded or multi-threaded, what the processor is that the code is running on, how many cores there are, etc.
C++ is an abomination. Every time someone writes a line of C++ code, a kitten dies.
Either write in C, or use a proper high level language.
Peace,
Cameron Purdy
Both are missing the point.
by Paulo Pinto,
Your message is awaiting moderation. Thank you for participating in the discussion.
Somehow all this discussion about native vs managed, ends up discussing C++ vs JVM vs .NET, while missing the point that before Java all the benefits of managed languages were already available in native compiled languages.
Herb focus only on C++ because it is what he cares about. Delphi, Ada, Modula-3, D are safe languages, with system programming capabilities, able to provide abstraction mechanisms similar to C++, while compiling to native code.
The best is to have an hybrid solution, similar to what environments like Lisp or Eiffel provide, by using a VM for quick turnaround development, with a full compile to native code for easy redistribution of binaries or environments where the usage of JIT is not possible.
Re: Welcome to 1995
by Paulo Pinto,
Your message is awaiting moderation. Thank you for participating in the discussion.
This is not completely true.
Most C++ compilers support for long time profile guided optimizations, which can help in the same way a JIT does. Grated you still cannot optimized to unknown architectures/instruction sets. But in .NET and JVM most special instruction sets (SSE, AVX,...) are also left unused.
Unfortunately C++ seems to be the only multi-paradigm high level language with native code generation implementations that is currently accepted in the industry.
Until a language with the same abstraction capabilities as C++, with native compiler implementation is accepted by the industry, C++ will continue to hold its position.
Re: Welcome to 1995
by peter lin,
Your message is awaiting moderation. Thank you for participating in the discussion.
Static profiling isn't the same as runtime optimization. In many situations, usage changes over time, so static profiling/optimization would need to be re-run. The types of optimization like SSE is a gap in managed runtime environments like CLR and JVM, but you can use native libraries in both. The reality is neither one solves all problems. The best approach is to use both where they are strong and compliment with other tools.
You can put lipstick on a pig
by Dan Tines,
Your message is awaiting moderation. Thank you for participating in the discussion.
But the performance trade-off (whatever that might be) isn't worth getting muddy in the pig pen.
Re: You can put lipstick on a pig
by peter lin,
Your message is awaiting moderation. Thank you for participating in the discussion.
which one is the pig? C++, managed languages or bad developers?
Re: You can put lipstick on a pig
by Paulo Pinto,
Your message is awaiting moderation. Thank you for participating in the discussion.
Personally I would say bad developers.
Test and Change
by Glenn Graham,
Your message is awaiting moderation. Thank you for participating in the discussion.
Personally I like managed code for two key reasons. I can integrate testing in more easily and I can work with the code (especially with changing it) more easily. Performance seems to be a small effect and now a days. I would rather multi-thread for performance and that is much easier for me in managed code.
Re: You can put lipstick on a pig
by Dan Tines,
Your message is awaiting moderation. Thank you for participating in the discussion.
"Modern" is the lipstick and C++ is the pig.
Space vs Time
by Russell Leggett,
Your message is awaiting moderation. Thank you for participating in the discussion.
I think performance is a red herring. We've seen that JITs can find a lot of optimizations that static compilers can't based on runtime observations, and there is a lot more for growth there than static compilation. The real trade off I see, is when those optimizations are made. Java requires a lot more memory than a C++ application, and can take a lot of time to start and optimize hotspots etc. This will likely always be the case. C++ does more upfront, and so it's fast immediately, but can't do additional optimizations during runtime.
Re: Space vs Time
by peter lin,
Your message is awaiting moderation. Thank you for participating in the discussion.
I would agree a good C++ compiler can optimize, but that assumes the code is well written. If it's spaghetti code that's poorly organized and badly implemented, no compiler switch is going to help. The same is true of managed environments. Atleast with JIT, really bad code can be in-lined based on runtime profiling. I've seen this first hand with really poorly written java code. The other big obstacle I see is skill. There aren't a lot of great C++ programmers that have elite skills. From my experience developing business applications, often the functional specifications are obtuse or wrong. Using a managed environment "can" make these types of apps easier to develop. On the other hand, if an application is clearly defined with very specific performance and scalability requirements beyond what managed environments can provide, the best choice is to hire a top notch C++ developer.
Re: Welcome to 1995
by Charlie Helin,
Your message is awaiting moderation. Thank you for participating in the discussion.
I can only speak for the JVM, but it does for instance utilize utilize SSE2 instead of x87 the vast majority of instructions. I wonder were you get your information from?
Re: Space vs Time
by Charlie Helin,
Your message is awaiting moderation. Thank you for participating in the discussion.
Peter,
It is true that the JIT/Optimizer will figure out the 'intent' of the program rather than trying to follow it as is. I believe that most programmers are to smart for their own good and waste huge amount of time trying to be cute and obfuscate their code to impress colleagues, or just because it looks more intelligent write their program as it was an unmanaged environment. In short making their code less readable.
This unfortunately sometimes makes the job harder for the optimizer to figure out for instance if it is safe to hoist the "out of bounds check".