Modularity Maturity Model
At the OSGi Community Event last week, Dr Graham Charters from IBM delivered a presentation entitled Towards a Modularity Maturity Model, a work-in-progress set of requirements for identifying the maturity of a system with respect to modularity. The intent is similar to that of the Capability Maturity Model, and provides a way of measuring how closely an organisation (or project) has advanced down the modularity path.
Note that the below is a work-in-progress, and the numbers/descriptions may evolve over time. With that caveat in place, the modularity maturity model currently looks like this:
- Level 1:Ad Hoc. Nothing is modular. Everything is a bunch of JARs, or worse, a bunch of classes. Typically results in a monolithic application.
- Level 2:Modules. Modules have a formal versioned identity, and dependencies are done against the module identity rather than the units themselves. Maven, Ivy, RPM and OSGi fall into this category.
- Level 3:Modularity. Modules are declared via module contracts, not via explicit module identifier/versions. The requirements might be abstract (e.g. Declarative Services is available) or might be specific packages (e.g. org.osgi.framework).
- Level 4:Loose coupling. Implementations are not found via factories or constructors; instead, they are queried dynamically from a registry or injected on demand.
- Level 5:Devolution. Artefact ownerships are devolved to modularity-aware repositories. They may support collaboration or governance for accessing the assets by relation to the capabilities required.
- Level 6:Dynamism. Modules participate in a dynamic life cycle which includes the ability to add, update and remove modules at runtime, whilst preserving the state in the system.
InfoQ caught up with Dr Graham Charters and started by asking him what drove him to create this maturity model:
Graham Charters: I've been fortunate enough to talk with a lot of customers over the past couple of years who are at various stages in their adoption of OSGi. They're typically doing modularity-after-the-fact, meaning they have some existing application or applications that have become unmaintainable and are preventing their businesses from adapting and growing. Often the customers feel like they're the first people to embark on this journey and it looks to them like a journey with no map and no clear destination. What the maturity model does is capture the most common and logical adoption journey I see today. I made it modularity technology agnostic because the lowest level does not require OSGi, although to my mind, OSGi is the best solution, but it also helps be clearer in explaining what the characteristics and benefits are of each level, without going into specific technical details of how to achieve it.
InfoQ: What is the expected (or observed) distribution of projects in this model?
Graham Charters: I did a survey at the OSGi Community Event that pretty much confirmed what I thought. The view from the room was that, as an industry, we average around a level 2, Modules. I also asked the question specifically about Eclipse Tools projects and the view was they were around a level 3, Modularity. There are projects that are at level 4 (Loose-Coupling) and properly exploit the OSGi services model, but they're often ones with a specific OSGi focus, for example, Apache Aries, Apache Felix, Eclipse Gemini, Eclipse Virgo. Devolution (level 5) and true dynamism in the presence of stateful services (level 6) are very rare. My hope is the up-coming OSGi Bundle Repository spec will begin to unlock level 5.
InfoQ: Is there a tendency to follow through these linearly? What about the last two (Devolution/Dynamism)?
Graham Charters: Generally, organisations will follow through the levels linearly. The order is essentially the path I see customers following most often, and is most logical, given the technologies available. That said, the order between Devolution and Dynamism is not particularly strong and is more a reflection of the distribution of customers I've seen who are interested in these levels. It is quite common for customers considering repository collaboration and governance whereas dynamism is far more specialised. I think there's probably a reasonably high correlation between dynamism and headless/embedded OSGi usage.
Another area in the model that could evolve over time is the ordering and distinction between Modularity and Loose-Coupling. Some would argue that Loose-Coupling, or implementation independence, between modules is part of modularity. To a certain extent, they're right. You can describe the sharing of implementations in terms of requirements and capabilities. However, oftentimes customers need to preserve running their modules in a non-modular environment and so adopting Loose-Coupling is problematic. This is where I see tools like micro Services (aka pojoSR) being useful. Technologies like this also have the potential to allow customers to swap the order or adoption of Modularity and Loose-Coupling. One thing that's often not appreciated about Loose-Coupling is its impact on the Modularity layer. If your modules are collaborating around something like services, you greatly reduce the API surface area between your modules because you no longer need to share implementation classes.
InfoQ: Are there cost savings to be made moving along the maturity model? Can it be arrived at incrementally?
Graham Charters: Yes, there are cost saving to be made and being able to clearly articulate this was one of the goals of the model. I wanted to capture in general terms for each layer, what the characteristics are that an organisation has to exhibit, i.e. what they must be repeatedly doing in terms of modularity, and what the benefits are. Moving up a layer has a cost in terms of adopting certain practices in development, build, and operations, but also has benefits. I characterise each benefit in terms of a decoupling, where each decoupling increases an organizations agility, and increased agility results in reduced costs and reduced time to value. For example, to move from Modules to Modularity (levels 2 to 3) an organisation must start describing their modules' externals and adapt their development, build and operations to be aware of this. The benefits are a far greater awareness of the structure of their systems, which reduces system rot and makes them more maintainable. You can immediately determine the impact of any module changes on the system because the modules can describe whether this is a non-breaking implementation change, a bug fix, an implementor breaking change, and so on. You can also be agnostic of module refactoring because module consumers no longer care which modules provide certain capabilities. Finally, you get early warning of any breaking structural changes because you immediately know if any requirements are not satisfied.
When organisations succeed in creating a place or places to collaborate around modules (contribute, review, discuss, etc), they're far less likely to create duplicate modules. This reduces cost and increases re-use. With increased re-use, comes increased quality. What organization wouldn't want their developers to be able to search for the APIs and services they need and then be able to pull down high-quality modules into their projects, rather than requiring them to find something of unknown quality or write something from scratch?
InfoQ: Are these measures specific to OSGi, or can they be aligned to other modularity frameworks?
Graham Charters: I tried to make the measures and benefits modularity technology agnostic. I did this for two reasons - the first reason was, I think it helps when explaining to a broader group what the characteristics are that need to achieved for a particular level. If an organisation is at level 2, Modularity, and they're not using OSGi, it doesn't make sense to talk about import-package and export-package, whereas talking about the benefits of defining module externals should be clear. The second reason is, I think it helps people when evaluating modularity frameworks. Not all modularity frameworks are created equal and so the model helps identify the desired characteristics and benefits and thus the appropriate frameworks to use.
InfoQ: Is there any plans on releasing this as a formal standard document through OSGi or another organisation? Are there expected time lines for this to happen?
Graham Charters: To be honest, I wasn't sure how well the idea would be received, and I've actually been overwhelmed by the positive response. I hoped there would be some people who liked the idea and would be willing to collaborate on its evolution. I'm not sure it's the sort of thing that warrants a standard, but I could see it evolving into a whitepaper, similar to the semantic versioning one published by the OSGi Alliance. Given the level of interest I would hope we could get a first pass published in the next month or so.
InfoQ: Is there anything we can learn as an industry and as an OSGi community from the maturity model?
Graham Charters: The view from the rather unscientific survey I did at the OSGi Community Event suggests that most organisations and projects are at level 2, Modules. Personally, I believe there are huge quality and productivity benefits to be gained from reaching level 5, Devolution. This is particularly true for organisations that have large distributed development and operations teams. If we make level 5 our goal, then we need to sell people on the benefits of the higher levels and make it as easy as possible for them to move up through the intervening levels. For example, to move to level 3 requires good tools for defining module externals, for performing continuous build, for module and system analysis to spot potential problems, and to aid with problem determination.