Can the Simple Module System save JSR294?
Over the past month there has been a lot of debate on the current state of the Java Modularity working group (JSR 294). Although the JSR tries to find common ground between different module systems (notably Sun's Project Jigsaw and OSGi), the current set of proposals are overly complex and introduce the world's first concept of a meta-module system.
Much of the debate has centred on the difference between a language specification and its implementation. As it currently stands, the JSR 294 draft (which is disliked by most of the expert group) defines a meta-module system which delegates the concept of modules, versions, constraints and dependencies to a 'pluggable' external modularity provider. Unfortunately, this means that there can be no common ground for version numbers, dependencies, and ultimately leads to the fact that modules developed for one or other of the module systems will likely be incompatible for the other.
Such vagueness helps no-one in the Java community, which is in dire need of a module system which can work with either the OpenJDK jigsaw proposal or OSGi. Many have asked why OpenJDK can't reuse OSGi – but the simple answer is that OSGi is much more than just modularity, and although the core spec (which defines the module layer) is fairly small, licensing issues are still a factor in why this could not happen in any case.
Instead of developing a meta-module system with vaguely defined semantics and the concept of external providers dictating the meaning, a Simple Module System proposal has been posted to the mailing list group suggesting that now is the time to get rid of the meta-module system, and focus on the commonalities that both module systems can support. This would involve coming to an agreement on some key differences in the way that module versions are represented (a win for Java developers everywhere, in any case) and define module dependencies as much like the Maven module dependencies, where a module implicitly imports all the packages it needs whilst exporting all the packages that it defines.
This would allow Java developers to build a module that works in both Jigsaw and OSGi, and though both have more powerful extensions, this wouldn't be needed for the majority of developers who build Java libraries and need to provide modules. This would avoid the need for providers to choose which of two otherwise incompatible formats to use and instead focus on static modularity of Java libraries. More powerful dynamic modularity could then be used by using those modules within a dynamic module system; but for most use cases, static modularity would be enough.
The key points of the proposal are:
- Simple Visibility Model – when a module requires another simple module, all the contents of this other module are made visible to the dependent module. There is no proscription against splitting packages across modules. This is what Jigsaw currently does and is what most people, unfamiliar with OSGi’s more expressive model, are likely expecting. It is very much what most people experience today with Maven based build systems and the current runtime classpath model. OSGi must update its specifications to provide support for this simple visibility model, which is almost identical to Require-Bundle on a bundle which exports all its packages.
- Single Version Numbering Scheme – A single, shared version numbering scheme is necessary. The simple module system must define a version numbering scheme and Jigsaw must use it and OSGi must change to use it. The version numbering scheme must define a natural ordering so that compareTo() can be used to order versions. Currently, OSGi defines a 4-part version numbering scheme and a natural ordering over it. Sun has stated the OSGi scheme does not meet Jigsaw’s requirements. These requirements must be collected in order to design a version numbering scheme and its natural ordering which can be shared by the simple module system, OSGi and Jigsaw. OSGi must update its specifications to support this new version numbering scheme.
- Module Membership – For the simple module system, the compiler must assume that module membership corresponds to the membership of the module artifact, for example a JAR file or directory. At runtime, each module artifact will have its own class loader that is associated with a single unique module. All types loaded from a module artifact will be loaded by this single class loader and made members of this single module.
- No “black holes” – In order for this proposal to be truly valuable, it is paramount to have no “black holes” in the specification, where black holes are things in the source code which can not be understood without the assistance of a specific module system. The specification must specify the concrete syntax and semantics of the simple module system including version numbers, dependency expression, module membership, etc. The Java language specification does not allow one to define their own language keywords or operators. The same must be true for module information in module-info source files and for module membership rules.
Due to the way that the JSR process works, one or a few people are responsible for writing the specification and the expert group is there to debate, point out issues, or approve the draft. For some time, the meta-module system hasn't met with common approval for the members of the group, and a number of the expert group members (including a Sun employee) have felt the need to write their own proposal in order to highlight the issues with the current draft. This feels like a last effort to try and get to a standardised module system for Java, and it's hoped that both OpenJDK's Jigsaw requirements and OSGi's can be met by a simple module mechanism which is defined in its entirety in the language specification without appeal to an external implementation.
Perhaps the biggest win would be an agreement on the representation of version numbers. OSGi has long defined a
major.minor.micro.qualifier format, which is sufficient for all Java libraries except Sun, who uses
1.major.minor.micro.qualifier. But the other significant difference is that the empty qualifier in OSGi is the 'bottom' value of the version number range, whereas in most Java developers common usage and in Jigsaw, the empty qualifier is the 'top' value. (In other words, 1.0.0 is bigger than 1.0.0.beta in Jigsaw, but has the opposite in OSGi.) Finding a common ground to solve this as a special case would be a good step towards the module system version requirements, and even if there's requirements for the addition of a fifth digit, it could be the case that a future version of OSGi would be able to support this.
What do you think of replacing the meta-module system with a simple module system for Java?
This has gone on more than long enough.
Re: This has gone on more than long enough.
I gotta say, this cracks me up.
Thus I would question the statement "sufficient for all Java libraries". There is at least thousand of NetBeans modules and for them the OSGi versioning scheme is not really sufficient.
PS: One can of course claim that NetBeans is Sun's project. Not fully and moreover our versioning scheme was in place before Sun bought us ;-)
But just reading the above 4 comments (no pun intended just an observation so don't take the following personal) I'm sure that this will never happen do to prejudice and closed mindedness (I need to protect my assets....) on all side.
major breaking change
minor backward compatible change
micro bug fix
qualifier build id
In a committees, it is always easy to just make it infinite and flexible to keep everybody happy. With a lot of flexibility, each has their own solution "standardized". However, I strongly believe that with a spec there is also a cost of making things flexible because the interpretation becomes harder. This cost needs to be balanced with the gain imho.
If you make the version flexible you loose the advantage of the semantics because the author of the version can put his private semantics on all the parts. This makes it impossible to interpret what a version means. In OSGi, at least you know what version 1.1 means relative to 1.5.7. For this reason, there are many tools that can interpret OSGi versions because of their simplicity and concrete, well defined, syntax.
The most frequent need for alternative version schemes I see is where the marketing department puts non-technical constraints on the versioning scheme.
That said, the OSGi Alliance is very technically focused organization. If there are sound reasons for a new versioning scheme, then I wholeheartedly support adoption.
If you want to know where the 1.major.minor.micro.qualifier comes from, read an old blog of mine: Is 9903520300447984150353281023 Too Small? :-)
splitting packages across modules is better
When a module depends on a set of packages it does not care which are the modules that provide them. In this case you are free to change the module structure without updating any dependent modules!
On the other side if you want to split a module into more modules or join some modules into a single one you should update all other modules that are reffering two those ones - very very bad.
In other words if you use a coarse grained dependency scheme (module dependencies) instead of a fine grained one (package dependencies) you are blocking refactoring on a module level!
Simple Visibility Model is bad for Module Refactoring
By Eric Evans
MODULES need to coevolve with the rest of the model. This means refactoring MODULES right along
with the model and code. But this refactoring often doesn't happen. Changing MODULES tends to
require widespread updates to the code. Such changes can be disruptive to team communication
and can even throw a monkey wrench into development tools, such as source code control
systems. As a result, MODULE structures and names often reflect much earlier forms of the model
than the classes do.
Inevitable early mistakes in MODULE choices lead to high coupling, which makes it hard to refactor.
The lack of refactoring just keeps increasing the inertia. It can only be overcome by biting the
bullet and reorganizing MODULES based on experience of where the trouble spots lie.
Some development tools and programming systems exacerbate the problem. Whatever
development technology the implementation will be based on, we need to look for ways of
minimizing the work of refactoring MODULES, and minimizing clutter in communicating to other
Shane Hastie on Distributed Agile Teams, Product Ownership and the Agile Manifesto Translation Program
Shane Hastie Apr 17, 2015