Q&A with Aleksey Shipilev on Compact Strings Optimization in OpenJDK 9
JEP 254 introduced the compact strings optimization for OpenJDK 9 with the goal of improving the footprint of Java String and related classes while not compromising performance or compatibility.
The compact strings optimization has made it into OpenJDK 9 build 93 and community participation in testing and reporting bugs is highly desired.
InfoQ took this opportunity to interview Aleksey Shipilev, a Java performance engineer at Oracle; to understand more about this optimization and its performance impact.
InfoQ: JDK 6 introduced the compressed strings optimization. What was wrong with JDK 6’s compressed strings that was the catalyst for the feature to be removed in JDK 7 (and JDK 8)? Is compact strings a build up on the compressed strings implementation?
Shipilev: UseCompressedStrings was really the experimental feature, that was ultimately limited by design, error-prone, and hard to maintain. Compact strings is the project that tries to reiterate that approach, but without the pitfalls of UseCompressedStrings.
First of all, JDK 6 UseCompressedStrings taught us that many applications enjoy 1-byte Strings, and the footprint/performance benefits are enormous. Therefore, we (JEP developers) were convinced that the "big picture" is correct, and we "just" need to work out little details.
Second, UseCompressedStrings feature was rather conservative: while distinguishing between char and byte case, and trying to compress the char into byte on String construction, it done most String operations on char, which required to unpack the String. Therefore, it benefited only a special type of workloads, where most strings are compressible (so compression does not go to waste), and only a limited amount of known String operations are performed on them (so no unpacking is needed). In great many workloads, enabling -XX:+UseCompressedStrings was a pessimization.
Third, UseCompressedStrings implementation was basically an optional feature that maintained a completely distinct String implementation in alt-rt.jar, which was loaded once the VM option is supplied. Optional features are harder to test, since they double the number of option combinations to try.
So, instead, you would like a feature that is:
- Enabled by default, so that it is always under great scrutiny both from correctness and performance aspects;
- Footprint improvements notwithstanding, provides equal-or-better performance on 1-byte strings, so that users would experience performance boosts from having less String data to process;
- Accepts only a moderate, if any, performance/footprint regressions on 2-byte strings, so that users would not get penalized for having enabled-by-default compact strings.
That's JEP 254 "compact strings". Of course that requires much more engineering work. You have to, at least:
- Rewrite Java parts to handle String operations in both 1-byte and 2-byte forms without repacking - this is crucial for making String highly performant;
- Teach VM parts that do interop with Java code to handle both String forms -- you know that VM can instantiate its own Strings, and also there are JNI interfaces that have Strings in them, right?
- Fix up and double up compiler intrinsics for both String forms, and sometimes even intrinsics that handle binary operations on different String forms - like String.compareTo(String);
- Fix up -XX:+OptimizeStringConcat code, that handles StringBuilder optimizations for string concatenation to accept both String shapes;
- Provide a "kill switch" to allow users to turn the feature off, if you know beforehand this feature would not benefit your cases;
And on top of that, verify both functional and performance metrics are not regressing. In many cases, a simple and obvious solution to the single task would get exploded into dealing with all sorts of codegeneration issues you have to either resolve or workaround to fit the performance goals.
But, all that will pay off at the end!
InfoQ: What approach did the team use to validate that no regressions were observed as far as throughput or memory footprint is concerned? Also, java.lang.String is a very pervasively used object; how can you begin to cover all the various different types or uses of Strings?
Shipilev: There are four things you have to realize while working on these features.
First, there is ultimately no way you can guarantee the absence of performance regressions even for small changes. With the enormous size of Java ecosystem, just by the law of large numbers, there would be scenarios where you will have an accidental (and frequently fixable) regression, and there are places where you will have to accept a known regression. This is mildly sad, but true fact about platform engineering in any large-scale project.
Second, understanding that first thing, you have to accept that you can make only so many tests, work through your own (practically limited) set of workloads to see if the feature behaves properly, and then put the release out for users to run their own applications, in their own scenarios, that we cannot even start to imagine. This is the rationale behind publishing Early Access releases. So please, it is terribly important that you test JDK 9 EA releases regularly, start doing it *right now*! This would greatly aid OpenJDK development, not to mention smooth your own transition to JDK 9 in the future.
Third, for some performance-sensitive classes, you have chicken-and-egg problem: the workloads you want to improve may have already worked around their performance woes -- the same woes you would like to optimize. For example, Lucene does their own data representation for a while now, do compact strings help Lucene? Of course not, in all the places those projects already have their own solution.
So, you end up rolling your own microbenchmarks that bash the particular parts of your new solution, optimize for them, realize what are pros and cons for either approach, understand the improvements/regressions you have, and then validate "real life" (whatever that means) workloads are either improving or not regressing. If they are, beyond what you expected, you update the microbenchmarks, and respin. After a while you will converge to some stable point, where your understanding matches all the experimental data, and that understanding matches the release goals!
InfoQ: What kind of analysis / data did you use to become convinced that implementing this feature would offer improvements in footprint reduction, yet preserve or maintain throughput and latency?
Shipilev: We have a large corpus of heap dumps from our customers' applications that can be used to estimate the heap occupancies for some frequent classes. So, we have crafted a simple simulator that introspected what Strings are there, and how much characters they consume, and how compressible those characters are. It was estimated that those application should experience 5-15% reduction in footprint.
Our larger validation runs after compact strings were implemented indicate we hit this estimate, sometimes with a room to spare!
Throughput/latency wise, we reused the experience from UseCompressedStrings that improved the performance on some targeted workloads. At the beginning, we were wondering if we can trim the footprint while improving the performance across most workloads, – I was even on record calling this a “scary improvement” – and as we did more and more prototyping, it became clear that we can.
InfoQ: If someone is interested in single byte character based String and want to write their specialized String class because it seems less risky; what would your advice be to them?
Shipilev: Well, UseCompressedStrings did just that, see the pitfalls above. Many people underestimate how much effort is required to make sure a specialized String implementation works fine in a wide range of scenarios. So my advice is “Don't”.
InfoQ: Any presentations on compact strings optimization that you would like to discuss?
Shipilev: Charlie Hunt did an excellent overview at QCON SF and JavaOne in 2015. I will do a hopefully more in-depth session at upcoming JFokus tech summit, and probably on virtualJUG shortly afterwards.