BT

The Last Frontier in Java Performance: Remove the Garbage Collector

| by Abraham Marín Pérez on Mar 01, 2017. Estimated reading time: 7 minutes |

Aleksey Shipilëv, performance and OpenJDK developer at RedHat, has filed a new JEP draft to create a no-op garbage collector; that is, a GC that doesn't actually reclaim memory. This collector is aimed at aiding JVM implementers and researchers and, to a lesser extent but perhaps more interesting for the public, ultra-performant applications that generate little to no garbage. If the JEP goes ahead, the new GC would be available together with the existing ones, and would have no effect unless explicitly activated.

Garbage Collection and Java performance is always a complex topic to handle, and to help clarify things InfoQ reached out to Java Champions and performance experts Martijn Verburg and Kirk Pepperdine. We also talked to Remko Popma, who leads the transformation to a garbage-free Log4j, about how this objective can be achieved. Verburg and Popma confirmed that, in their view, the main beneficiaries of a no-op GC, or Epsilon GC as it has been called, would be GC developers and performance researchers. Epsilon GC can serve as the control variable to measure the performance of other garbage collectors. As a simplified example, we could have an application running with a no-op GC for a reduced GC overhead (barring other considerations about memory allocation and mutation control). If the same application is then run with the same workload but with different GC algorithms configured, the difference in performance would indicate the impact that the garbage collector has on the application. This will help GC developers and performance researchers understand the behaviour of garbage collectors in a more isolated manner.

"I think that this is actually a great step forwards for allowing more accurate benchmarking of various parts of the JVM (such as the existing JIT C1/C2 compilers, the possible shift to Graal, etc). It will really add extra longevity for the JVM." Martijn Verburg

On the other hand, ultra-performant applications could benefit from Epsilon GC. There is a rare breed of applications and libraries, like the aforementioned Log4j, that have been implemented in such a way that they produce no garbage, and therefore have no need for a garbage collector; for this type of applications, performance could be improved by removing the overhead of the collector. However, as Popma highlighted, building a library that can run with Epsilon GC "would take significant engineering effort to make sure an application's memory is managed carefully enough that it won't run out", and even then a risk and benefit assessment has to be made to ascertain whether the gains obtained by choosing a no-op GC are consistent with the difficulty of achieving a zero-garbage state.

It might seem difficult to envision how an application can be written to produce no garbage however, and although the topic is much more complex than what could be explained in this article, it might be easier to understand by taking into account the following considerations:

  • Memory is managed through two different mechanisms in the JVM: the heap and the stack; this is why there are two different errors regarding a lack of memory (OutOfMemoryError and StackOverflowError). Memory placed on the stack is only visible by the current thread and during execution of the current method; therefore, when the current thread leaves the current method, this memory is automatically released without the need of a garbage collector. Memory in the heap, however, is accessible by the entire application at any point, which means an independent garbage collector needs to verify when a piece of memory is no longer in use and can be reclaimed.
  • Allocation of primitives always goes on the stack, and therefore poses no pressure for the garbage collector. If one was to write code using mostly primitive types, there would be fewer objects for the garbage collector to look after.
  • Producing no garbage is not the same as producing no objects; objects can still be created without the need of a garbage collector to look after them:
    • An application or library could produce a number of objects at the beginning and then reuse them constantly; this requires the developer to understand the memory footprint of the application very well.
    • Sometimes the compiler can understand that a particular object is not going to be used outside of a method; this is known as escape analysis. When the object is found not to escape the method, it can be allocated to the stack as opposed to the heap, and therefore eliminated automatically as soon as the current method finishes.

Although all of this is possible, Pepperdine pointed out that this is a very unnatural way of writing code that implies losing many of the benefits that Java provides, while Verburg also indicated that memory management is precisely one of the main reasons why Java has been so successful in the industry. In addition to this, we need to bear in mind that, despite its name, the garbage collector doesn't only have the task of reclaiming unused memory, but also allocating new blocks, and Epsilon GC would still have to do this. Pepperdine uses this argument to suggest that, at least in theory, there would be no significant difference in performance between having Epsilon GC and having any other garbage collection algorithm tuned to the point where it doesn't actually do any garbage collection.

However, even considering all these caveats, both Pepperdine and Verburg confirmed that there is a small audience for which this kind of behaviour could be very useful, but the fact that this audience is small poses, according to Kirk, doubts about its usefulness. The vast majority of applications out there need to reclaim memory at some point, and therefore needs a functional Garbage Collector.

"Reasonable GC pause times are simply not an issue for most applications, so why give up all the benefits of Java for a questionable performance win." Kirk Pepperdine

Pepperdine also reminded us that every new feature adds maintenance costs to OpenJDK, and therefore OpenJDK developers should take into consideration the big picture when adding them. Oracle has been reducing the number of available garbage collectors precisely to reduce maintenance costs, and therefore adding a GC that will only be useful to a small percentage of users may not represent the right investment. Shipilëv did seem to consider these points when filing the JEP draft though, and the preliminary analysis seems to indicate that the overhead would be minimal in this case, judging by the contents of the JEP draft and by the prototype that has already been provided. In fact, both Pepperdine and Verburg pointed out that, given Shipilëv's experience, the fact that he is leading this iniatitive is reason enough to be optimistic about it.

On the other hand, Pepperdine also emphasised the fact that, although OpenJDK is the reference implementation of the JVM, there is no requirement for compliance of the garbage collector, which means vendors can implement their own algorithms and still be fully compatible with standard Java. This could cause a divison of opinions among the public: while some might think that implementing algorithms for niche markets may be something more appropriate for commercial versions of the JVM, others might consider this a very useful addition to OpenJDK.

Popma also recommended at least considering the use of commercial JVMs when performance becomes critical, since the cost of a license of one of these products could, overall, be lower than the cost of having engineering staff selecting and tuning a particular GC algorithm. However, even if someone was fixed on using only OpenJDK technologies, both Popma and Verburg mentioned the currently-in-development Shenandoah GC, which aims at producing ultra-low pause-times for very large heaps (100GB or more). In either case, the consensus among the experts seemed to be that, when it comes to application performance, a carefully-selected GC algorithm will almost always be better than no GC at all.

It is still early days for this proposal, and it needs to be reviewed and polished before it even becomes an official JEP. When and if this happens, a target version will eventually be added to it. Although at this time we can only speculate about what this target version might be, Verburg thinks that it is reasonable to expect Epsilon GC to be ready for Java 10 or 11. If anything, it would at least help understand what the interface of a GC should be, contributing to a more modular JVM.

Rate this Article

Relevance
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

This is a great idea by Will Hartung

It's a completely reasonable suggestion. It's a simple idea, and isn't asking for anything else, such as semantics and changes to allow manual memory management, hints for enabling stack allocation, etc. etc.

It simply suggests a NOP Garbage Collector. Now, the amount of work that may be necessary to get a Java program to not leak memory, especially utilizing the JRE libraries, is a completely different issue. But that's for another time.

It's a great idea. I'd never use it, myself, but it's a great idea.

This reminds me the Unistack NFX Pile by Néstor Sánchez

Which is a memory manager without GC, based on in-memory serialization/deserialization: www.infoq.com/articles/Big-Memory-Part-2

Re: The reactions when you suggest to put an option to disable the GC is te by Nitsan Wakart

what reactions? where?

what would that change, what would be gained ? by guillaume peron

where would cpu would be saved ?
Less tests to park code ?
Smaller object reference memory footprint if objects are not moved during GC ?
How does that compare to a well coded app with no full GC where objets living for a long time are not scanned by GCs ?

Re: what would that change, what would be gained ? by Peter Veentjer

At least one thing you will get rid of are 'write barriers' for card marking.

psy-lob-saw.blogspot.bg/2014/10/the-jvm-write-b...

Re: what would that change, what would be gained ? by Abraham Marin-Perez

Figuring that out is part of the objective of Epsilon GC. Kirk Pepperdine also postulated that, in theory, a no-op GC would be no different to a GC algorithm tuned to the point that it produces no garbage collections.

However, and as Peter Veentjer mentions, there are other considerations like write barriers (his link is very informative). And, finally, even if the application has been tuned so a STW garbage collection doesn't occur, the GC doesn't know that, which means it still needs a thread scanning and marking objects; if you remove that thread, there might be some extra performance that you might be able to gain.

Re: what would that change, what would be gained ? by Kirk Pepperdine

There are 3 considerations here, allocation and mutation managed by mutator threads, and collection managed by garbage collector threads. That we're talking about mutator threads as in more than 1, implies that we need these write barriers to ensure these threads to maintain a consistent view of memory. There maybe additional barriers that could be eliminated by the removal of the collector. For example the barrier for card marking to record the mutation of a pointer from tenured into young. However that mutator would *still* need to pass through write-barrier to expose the mutation to all other threads.

Escape analysis already minimizes the load on mutator threads so it seems to be the only way to eliminate any more of these barriers is to either improve escape analysis or limit the JVM to a single mutator thread.

As an example, I worked with a client to reduce allocation rates in their product. In our bench we went from 1.8GB/sec to 0 with a corresponding increase in throughput from 400,000tps to 25,000,000. While it would be interesting to run this benchmark using a no-GC JVM, I'm fairly confident that if there were any further gains realized, they would be quite minimal.

Re: what would that change, what would be gained ? by Kirk Pepperdine

After a conversation with Aleksey it got sorted that there are indeed gains in application throughput that may be realized with a no-GC version of the JVM. Would fun to see if these gains could be realized.

Is this telling you something? by Barry Smith

I can't help thinking that if you are trying to write Java code that doesn't create any objects on the heap then you really need to think about switching to a different language. Rust springs to mind.

Re: Is this telling you something? by Cameron Purdy

The challenge with adopting Rust is that nobody wants to be the first to adopt it. I wanted to try a project with it last year, and it really wasn't ready for prime-time. I wanted to like it ... (or at least, I wanted to avoid C++ ;-)

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

11 Discuss
BT