Is Project Jigsaw Back On Track?

One thing I can't understand: we have OSGi since 1999. It's a mature and proven technology now.
Why can't Oracle simply adopt and evolve it?

NIH syndrome

No, it is not "NIH syndrome".

Making the Java platform itself modular is a much lower-level implementation than OSGi, even if both Jigsaw and OSGi happen to use the same word "modularity".

If Jigsaw delivers on its promise, then it provide valuable low-level capabilities that higher level technologies and frameworks like OSGi can leverage.

Remember, building something into the JVM and the Java platform is a bit different than starting with the JVM and the Java platform, and then building functionality on top of it. One of the purposes of modularity in the Java platform is to be able to deliver different -- and obviously smaller! -- sub-sets of the Java platform without breaking existing applications; adding OSGi to the Java platform would not even begin to solve that challenge.

It is a disservice to position Jigsaw as "competing with" OSGi. Each has its distinct purpose, and there will likely be value in the combination of the two.

Peace,

Cameron.

For the sake of full disclosure, I work at Oracle. The opinions and views expressed in this post are my own, and do not necessarily reflect the opinions or views of my employer.

Cameron, would you say my characterization was incorrect, that there were originally two parts to Jigsaw, one "roughly" competing with Jigsaw (i.e. allowing projects to modularize into micro-service bundles) and the other dealing with the modularization of Java itself?

If so I would appreciate any explanatory description or link, and I can correct the wording. I believe though that's the common perception, that Mark had been promulgating until now.

Cheers
Victor

I can't speak authoritatively, but I'm certain that there have at times been ideas of how to provide some of the functionality of OSGi via Jigsaw, so it is correct that some people (including a few working on the Java platform) have thought of this as a competitive situation. Nonetheless, the strategy that has been clearly and repeatedly described by management within the Java platform group has been much more focused on providing enabling technology in the platform, and not trying to compete with or displace OSGi. I still believe that there will exist some overlap, but that the value of having modularity baked into the JVM and the platform will ultimately help OSGi, not displace it.

Peace,

Cameron.

NIH syndrome

I'm no expert, but I'll give it a shot...

1) This is exactly the problem that OSGi was designed to address; the end of JAR hell.
2) Not sure what you mean here. OSGi is just java with a little extra meta-data. Coupling OSGi with maven makes working with runtime and buildtime dependencies pretty painless.
3) OSGi does this already.
4) The Spring guys started an OSGi bundle repository and many artifacts in central are also OSGI-bundlized.
5) Maven again solves this, but OSGi provides support via declaring bundle version dependencies.
6) Proper architecture of your application into API and implementation alleviates much of this. If Jigsaw brings private packages, then I'm sure OSGi would use them.
7) Not relevant IMO. Oracle could deprecate java.util.Date anytime they wanted. With the new JSR-310 API in java 8, they arguably should. Just because something is in the JDK doesn't make it the best solution. Look at java.util.logging, for example.

For modularizing the JDK, I'm totally for Oracle taking the reigns. For providing the ability to create modular applications, I'll stick with OSGi.

Hi David -

"For modularizing the JDK, I'm totally for Oracle taking the reigns. For providing the ability to create modular applications, I'll stick with OSGi."

Yes, that is roughly the path that I was trying to describe.

Peace,

Cameron.

1) Right, but ultimately OSGI is a hack because it works off the jar manifest file, with versioning determined by parsing text. So jar hell still exists.
2) One example I heard of was a company selling financial auditing software to a company to ensure compliance with regulations, then distributing a jar that they could put in their classpath that would effectively replace the "scan financial data to ensure compliance" jar with an "all clear" jar that implements the same APIs. No link, sorry.
3) Yes, but this is not native to the Java platform, read my point 1) above.
4) You see where you're going, right? Instead of one company with a unified platform, you have the JVM vendor, the "module vendor" OSGI, and another party, Spring Source. Can you say "systems integration nightmare"?
5) See point 1) about text parsing hack and point 4) about systems integration.
6) It doesn't come anywhere near to solving the problem. If I need to share a class/method between packages I need to share it with the world. Epic fail! In Ceylon you can declare a package to be not shared (private), all of its methods and classes can be shared with all other packages within the module, but nothing in that package is exposed outside of the module. Only classes/functions/etc within a package marked shared are visible outside of the module.
7) Sun has deprecated much of java.util.Date since Java 1.1, but they can't eliminate those deprecated methods due to backward compatibility. With a module system, Oracle could eliminate that entire class and java.util.calendar from the main system libraries.

I'll add another point that I should have added earlier:

8) Other JVM language can benefit from first class module support in the JVM, instead of adding a fourth or fifrth layer to the systems integration nightmare you described above.

I'm surprised you had no thoughts on the link I posted with my first comment.

I'm no expert, but I'll give it a shot...

1) This is exactly the problem that OSGi was designed to address; the end of JAR hell.
2) Not sure what you mean here. OSGi is just java with a little extra meta-data. Coupling OSGi with maven makes working with runtime and buildtime dependencies pretty painless.
3) OSGi does this already.
4) The Spring guys started an OSGi bundle repository and many artifacts in central are also OSGI-bundlized.
5) Maven again solves this, but OSGi provides support via declaring bundle version dependencies.
6) Proper architecture of your application into API and implementation alleviates much of this. If Jigsaw brings private packages, then I'm sure OSGi would use them.
7) Not relevant IMO. Oracle could deprecate java.util.Date anytime they wanted. With the new JSR-310 API in java 8, they arguably should. Just because something is in the JDK doesn't make it the best solution. Look at java.util.logging, for example.

For modularizing the JDK, I'm totally for Oracle taking the reigns. For providing the ability to create modular applications, I'll stick with OSGi.

I agree with Cameron here, and I say that as an OSGi advocate.

The JRE itself has some pretty unique constraints that make it tough to adapt to OSGi directly, but OSGi remains the best solution for libraries and applications.

I am a little worried that the language from the previous revision of the Jigsaw requirements -- i.e. that it must interoperate with OSGi and that such interoperability must be proven by prototype -- has now been dropped from the latest requirements published by Mark Reinhold. I'm prepared to give Oracle the benefit of the doubt and assume that this was nothing more than an oversight, but I would very much like to see the document revised again soon to make it clear.

Insisting on build/runtime consistency of dependencies is a massive fallacy. It is explicitly NOT a requirement of OSGi, in fact the opposite is true. We compile against specifications but run with implementations.

Attempting to lock down the runtime environment such that it looks exactly like the build environment is the source of great fragility. The goal of OSGi's dependency system, version ranges and service registry is to allow us to build adaptable, reusable and future-proof software.

Do you have real examples of where "the runtime environment such that it looks exactly like the build environment is the source of great fragility"? That's a bold claim.

Furthermore, if I extend your argument to its logical conclusion, then Gavin King and the designers of Ceylon, as well as JetBrains (Kotlin), should not have introduced a first-class module system but should have interoperated with OSGI instead. If that is your argument, I find it impossible to accept.

Insisting on build/runtime consistency of dependencies is a massive fallacy. It is explicitly NOT a requirement of OSGi, in fact the opposite is true. We compile against specifications but run with implementations.

Attempting to lock down the runtime environment such that it looks exactly like the build environment is the source of great fragility. The goal of OSGi's dependency system, version ranges and service registry is to allow us to build adaptable, reusable and future-proof software.

There's also one question I have for the pro-OSGI camp:

Are your reservations against Oracle introducing Jigsaw due to the concern that Jigsaw will not come with OSGI compatibility, thereby breaking existing projects, or is the argument that this is a wasted effort on Oracle's part and Oracle should instead adapt OSGI for Java modularity?

Honest question.

The first is certainly a concern. If Java 9 with Jigsaw breaks OSGi-based applications then Java 9 must be considered a non-backwards-compatible release. There is nothing about OSGi that is non-standard Java, so if OSGi is broken then many other things could also be. Fortunately I don't think this is likely to happen, at least not intentionally. Just in case, the OSGi Alliance has set up a continuous integration server that runs the OSGi TCKs on top of the latest HEAD from the Jigsaw prototype, so that problems like this can be caught as early as possible.

Your second suggestion is less of a concern. I don't care if Oracle wastes their own internal effort -- I'm not a shareholder. However I do care if Oracle promotes Jigsaw as an alternative or competing module system for libraries and applications, especially if the end result of Jigsaw looks like some of the designs we have seen before.

As I've said, there are some unique problems in the JDK which makes it very hard to modularize in a normal way. A module system designed for this particular use case is not necessarily a good module system for everything else -- as they saying goes, "hard cases make bad law". If Oracle wants to create a module system for the general case then I have no problem with that so long as it is actually better than OSGi.

Let's say you write a component that wants to use JAX-RS. Since you plan to use CXF in the final application you compile therefore against CXF. This will give you the following problems:

1) It is easy to lock into quirks and vendor specifics
2) It is easy to pick up a dependency on on of CXF's transitive compile dependencies (Spring!)
3) A new release of CXF (or one of its transitive dependencies) forces you to re-release, even if it is just a minor bug fix unrelated to the API. In maven, this can be a total pita.
4) Switching to Jersey or another impl. requires a re-release
5) The final application has a significantly larger number of version constraints due to the CXF dependencies we bring to the table. The complexity of version constraints increases exponentially with the number of dependencies so this can turn out to be pretty bad.
6) Your component is highly successful with your boss, unfortunately, her application picked Jersey, so you're component is eligible.

In contrast, in OSGi you compile against the lowest versioned JAX-RS API that provide the functionality you need; in runtime you get bound to the highest possible implementation (CXF, Jersey, RestEasy, Restlet) that is compatible with your component and the rest of the application.

So now you get the following advantages:

1) No releases caused by changes in dependencies that are not visible to the component
2) Flexibility to switch implementations in the application
3) Many fewer version constraints on the application because your component only depends on JAX-RS.

So the reason you do not want fidelity between builds and runtime to develop (reusable) components is that it introduces unnecessary constraints on the application that uses those components. In reality, for larger applications it quickly becomes impossible (and that is why so many applications have to fudge their class path).

Build time fidelity with the runtime is like rigidly connecting the wheels of a car with an axel. Yes, it is very simple and it works perfectly; until the car has to turn a corner and breaks spectacularly. If you want to prepare for the future, then you better put in a differential.

Personally, my own concern about Jigsaw (the part that allows me to modularize my projects) is maturity. OSGi has been around the block for ten years, they already fixed mistakes years ago that Oracle has not yet even dreamed of.

So it is really not 2016 we are talking about. It is probably Java 10 or 11 before all the tweaks are just right.

Gosh my grandchildren will be continuing this debate by then!

First of all, Oracle can (and probably already has) mined OSGI for low-level requirements that the OSGI people had to figure out the hard way.

Second of all, new JVM languages like Ceylon and Kotlin come with modularity built in. Oracle can look to those languages for inspiration as well.

So I don't buy the argument that Oracle will "screw it up" because "it's new". Remember, these are the people that designed Java 8 lambdas not only as a language feature, but as part of a new collections API as well as some pretty well conceived JVM enhancements (invokedynamic and method handles).

The design documents for Jigsaw are pretty detailed. Oracle and Sun have had a lot of time to think about this.

I will accept the argument that it will take forever for them to do this. There are four phases to this project and only the fourth will see end user Java developers able to modularize their own projects outside the JDK.

Personally, my own concern about Jigsaw (the part that allows me to modularize my projects) is maturity. OSGi has been around the block for ten years, they already fixed mistakes years ago that Oracle has not yet even dreamed of.

So it is really not 2016 we are talking about. It is probably Java 10 or 11 before all the tweaks are just right.

Gosh my grandchildren will be continuing this debate by then!