Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Interviews Patrick Curran and Geir Magnusson on the Standardization Process

Patrick Curran and Geir Magnusson on the Standardization Process


1. Geir Magnusson: Hi, Patrick. How are you? You had a panel this morning that I couldn't make, but I did want to, so what happened?

Patrick Curran: We discussed the big question "What is the effect of open source on the standardization process? Does open source mean that there is less need for standardization? Does it replace it in some way?" Some people seem to think that that's the case. You get a bunch of people together and develop something collaboratively that somehow this is more Agile, that you can produce something faster than you can. People are always talking about standardization being very slow, very bureaucratic, time-consuming. I always like to explore those kind of questions.


2. GM: Did you come up with an answer?

PC: My own opinion on the matter is that the 2 are more complementary than in opposition. I think that there is significant value in process of standardization, by which I mean the collaborative development of specifications for interfaces or protocols or whatever. I think there is great value in doing that process. It enables the creation of multiple implementations of these technologies, it allows for interoperability, interchangeability - the old analogy with physical hardware where you can plug things into sockets and you know that it's got to work, wherever you are. All of that stuff is very valuable, but I think there is a lot of truth to the criticisms that we, in the standardization process, have been slow.

It can be very slow, as you know, it can be very tedious work and I think there are a lot of lessons that we can learn from the open source movement, in terms of getting a lot of people involved in the process, in terms of quickly iterating your way through a development process and so on. I think one of the conclusions we came to was that there is a danger in trying to standardize too soon. Depending on where you are in the technology curve, it can be a mistake to try and nail something down too early.

There is also a danger and a bunch of people sitting in the room and thinking and theorizing and saying "OK, this looks like a good design or a good set of APIs without the hands on experience that you get of actually creating something and having people touch it and use it." My take on it is that we got a lot to learn from and a lot of benefit from the open source processes in terms of collaboration, in terms of agility, in terms of transparency, the licensing models and so on. I'm constantly looking for ways that we can actually merge the 2 sets of processes, make the standardization process more effective by adopting the lessons that we can learn from the open source movement.


3. GM: It's funny how we can play open source, which is really about copyright - isn't it? - to the community dynamics that happen around a lot of projects that are open source. There are certainly open source projects that don't have those open transparent dynamics. There may be somebody throwing open source software license over a wall, so yes, I agree. I think there is a lot that you could learn or we could take from these open communities and how they work.

But, I do think standardization is important and it would be interesting to see if the collaborative development of open source communities and open source development, of having a neutral place where people can come and participate and build and see if it works - it would be useful. I've seen companies do this that want to collaborate with each other on a project, yet don't want to get hung up in IP trading, cross-licensing things to each other, have done it in say the Apache license, where there is a very explicit patent grant for the receiver of the software, there are lawyers who understand it and are able to work together and produce an Apache license body of code that either one can go and do what they wish with.

Apache license certainly wouldn't be the only license you could use. There is lots of good licenses: I mean GPL V3 has a clear language about patents there, too.


4. GM: In terms of the rejection of standards, was there any feeling about these being orthogonal things that you couldn't have open source on standards or the people realized that clearly open source is capable of implementing standards very well?

PC: Difficult to say! I think there is definitely a tendency for people to think that the standardization process is very heavyweight and possibly even unnecessary. Some people were saying things like "Things can emerge out of the communal activities of the developer communities". One point was made that if we standardized by looking for things that are already in existence and are working well, then that can be more effective than starting with a blank sheet of paper.

I have some sympathy with that position, but, at the same time, I think that, if you simply look for something to emerge out of the marketplace of ideas or whatever, you're not necessarily going to come up with something that is the most flexible, that is going to meet the needs of a broad community. It may simply be that there is a particular player that is particularly dominant, as a dominant market position or whatever, rather than something that meets everybody's needs.


5. True. It does get back to that question of if there is an implementation that's widespread, what are you doing? If it's a standards body, aren't you back in the room with the white board and the theorization?

If there is something that exists and meets people's needs, is broad, is general, then simply reinventing it for the sake of reinventing it, doesn't make a lot of sense. We talked about this within the JCP, about whether we should try to come up with some mechanisms for - I don't necessarily want to use the word rubber stamping, but - endorsing something that is out there and that is good enough. I certainly wouldn't advocate throwing it away and starting again just for the sake of doing so.


6. GM: Then you can come after the fact and write down what happened and say "This is the standard"?

PC: I think so. In a sense, some of the better work that we've done has come out of a process like I think of Doug Lee's concurrency work - JSR 166 - where they had a real open source community, they had 4 actual implementations before they chose to bring it and put it through the standardization process. Then, it can go relatively quickly. We had Steven Colburn on the panel and he talked about the process of starting with Joda Time and open source project for date and time libraries. He is now bringing that with the JCP and he points out that even though you may have gained considerable experience over several years in putting together a library, there can be a benefit to stepping back and say: "If I got a chance to do it again, how might I change it?" So, you got to kind of strike that balance.


7. GM: Was there any conclusion of these at all?

GM: In that sense, there is a project in the community that I've been following - The Open Web Foundation - which is very similar along these lines of giving things that have emerged from out of the market. What do you do to make sure that you can identify these as, if not a standard, a specification? There certainly is a difference between what you may consider an international standard and something that is clearly specified. This is really about specification. The best thing that came out of this is intellectual property protection. Being able to write down what you are doing, but also to get a bunch of organizations that might have some intellectual property patterns related to it to formally non-assert that, if you use this in this way, then you are free of risk from our patent litigation, which I think is a healthy thing. You get a virtual cycle - you get that far and you can get other people that are interested in the technology to use it, which further strengthens the use case.

PC: Yes, I agree that the intellectual property aspects of these, the protections that you get by engaging this process are one of the most valuable benefits. Going back for a moment to what you were suggesting, the difference between creating a specification and a formal standard, we did sort of touch on that. Clearly at a bare minimum, you need the specification with also pointed out that within the JCP we require conformance tests, which is extremely valuable and their reference implementation.

But, we did sort of touch on the question of if you've got that far, then why do you need to go the additional step and actually get the seal of certification or approval from a formal standards body? One answer to that is simply that there are certain organizations for whom that would be a requirement. Certain governments, certain large institutions may feel more comfortable in adopting or implementing technologies that have had that formal seal of approval of international standardization. Although it was suggested that things are changing and that we're still in the middle of it at the early stages of this transition and it's possible that people will feel increasingly comfortable with adopting something so long as the source is available and so long as it is well documented.


8. GM: You do sort of wonder what makes that group, the recognized international standard, and that group, not one of them. I think, in some sense, that's a bit of feeding energy to the monster, so to speak. We all recognize that group as an international standards making group, so it is. It really is about human labeling, isn't it? There is no supernatural being or daddy that comes down and says "Thou are an international standards body"

PC: There are 100 and some years of history here, so I think you can actually recognize some characteristics. These bodies tend to be endorsed by governments, receive the packing of government institutions, they tend to be genuinely multinational and, I guess, have earned the respect over some period of time - International Telecommunications Union Forum, whatever it was, 1840 something and still at it today.


9. GM: That may be just it, though. You will earn the respect and, at some point, people will open the circle to what can be included as a recognized standard, especially as the Internet makes the ability for this international participation so much easier. You have that formal representation and you have to have your own government send the representative body.

PC: We did touch on that also. I was suggested that, particularly with certain smaller specifications that there could be a more informal way of developing them that may be - I don't want to call it ad hoc - but in less formal process you can actually publish something. As you say, publish it internationally, anyone in the world can access it and can read it, if it is small and concise and easy to understand and people pick it up and use it, then it may well be that it ends up with almost as much moral authority as something that was stamped by the ISO or IATF or whatever.

GM: I think it's something in between, right? I probably myself have trouble recognizing this as a standard, but I think you can have a lighter way to organization. I think some of the key things are if the standard conforms to certain things with regard to intellectual property, that the process in which it was created has been transparent enough so that you can be sure that's it's not stalking horse for somebody doing evil, whatever that means.


10. I think it's something in between, right? I probably myself have trouble recognizing this as a standard, but I think you can have a lighter way to organization. I think some of the key things are if the standard conforms to certain things with regard to intellectual property, that the process in which it was created has been transparent enough so that you can be sure that's it's not stalking horse for somebody doing evil, whatever that means. There might be half way between where I think you do need a process, honestly. Formalization is good, it helps you trust the things were done in a way. If you trust the process and you can say that this thing was constructed according to that process it's much easier to accept things without having to investigate each one of these, right? That's the idea of a standard organization - that you trust how the process works. You at least know certain things about a specification even without knowing or participating in the spec.

PC: Right. Of course, you can hardly expect me to disagree, since our responsibility is to run the JCP and try to make sure that we do things in the right manner - trying to make the whole process more lightweight. Another of the themes that I tried to address when I attended conferences like this is "How can we get broader participation?" I often ask developers "Do you get involved? Do you read the specs? Do you contribute?" and the answer is often "No".

I don't know how we can deal with that. We've seen a small change recently. We offered free membership to Java user groups. We've had about a dozen join just in the last couple of months and I think it's going to be interesting to see what comes out of that because these are, for the most part, individual developers obviously committed because they put their energies into participating in Java user groups on their own time. It will be interesting to see whether, as more of these groups join the organization, whether that changes the dynamics of things, whether if that means we get more participation. I hope so, because I think we could use more committed participation on the part of individual developers as well as the large corporations and the non-profits like Apache and Eclipse.

You know that we are having big discussions within the executive committee about the extent to which we should encourage or possibly even mandate the use of open source processes and open source licensing for the work that we do. A survey that we did recently shows that about 50% of all JSRs that are currently active claim to be using open source processes and/or licenses. I understand they are very different, but a significant number are using open source licensing.


11. Do you think that there is benefit in doing so? Do you think that we should actually be mandating that, particularly for the reference implementation?

GM: It certainly it is a benefit. I think that the more open source implementations arise, the better. I don't care what license you put them under, as long as you don't mandate the license, I think this is better. The world's a better place is these RIs are open source. The worry I have is that forcing people to put a reference implementation under open source changes the dynamics about an organization it's required to make the investment, because as a spec lead you are required to deliver a working reference implementation, you are required to deliver a TCK. If you are in a commercial enterprise it will cost you money, it will cost you opportunity cost, it will cost you direct cost of having the people do it.

Maybe this is sort of 5 year old thinking and maybe organizations will be more than happy to do it this way, but if it would discourage people from participating - that could make a valuable contribution in terms of specification around some certain area - just because they couldn't afford to do it that way, they couldn't try to recuperate costs by incorporating the reference implementation into a product. That said, you could use a license which tends to discourage commercial - I don't use the word exploitation -, but if you use a copyright license that would prevent you from redistributing the software in the commercial terms, that might be a good half-way point. I still would be very much against mandating one open source license for this because I think that people should be able to choose.

PC: It may be that if we do indeed move in that direction, that would require us to move away from the model where the expectation is that all of the work for doing the RI - reference implementation - is done by the spec lead company and maybe that would have to become a collaborative effort in the sense of open source development process.


12. GM: And you force it. As the spec lead signs up and says "I promise to deliver this", that has a forcing function that that's what the spec lead is supposed to do, but as an expert group member, what would you say? That you are required to deliver a tenth of it?

PC: It works for many open source projects, right? People collectively build stuff and deliver it.

GM: Yes, but they were very motivated to do so. It's generally on volunteer basis. Certainly, the Apache projects are all volunteers. I mean people are paid to work on it, but they are paid for by their employer. That's really an orthogonal concern of the ASF - "We don't care if you are being paid for, everybody's there individually as equals and working on it because they have some motivation to do so", whether it's your employer, whether it's you are just interested in the technology, whether you are learning - it doesn't matter!

The difference, I think, is that open source projects do not have a required delivery. They can go on for quite some time without formal release or a release version or a complete version. They also are not required to have any convergence, necessarily, where a reference implementation of a spec would better convergence to an implementation of the spec or else you have a real problem. I wonder if this idea would result in even longer timelines for completion of specs.

PC: I guess that's possible. Nevertheless, what we are clearly seeing is a strong trend in this direction. Certainly, for EE, I think it's the majority of JSRs are now being done this manner. SE is kind of half way there, ME is where the business model seems to be different, and so far. At least not such a broad embrace of open source, but I think it's changing and I think we are going to see significant movement in this direction over the next few years.


13. GM: Do you think what Google's doing with Android is going to help drive some of that openness back into the ME ecosystem, when you can have this whole family of devices that for the most part are open in their software platform?

PC: I think that has to happen. I was in Barcelona for the Mobile World Congress just a few weeks ago and one of the major themes there was openness. The word "openness", as you know can mean many things. In that world, there are additional meanings that it takes on, but somebody explained it fairly simply "Compare and contrast the mobile phone world with the PC world. In the PC world you can buy a hardware from one manufacturer. For the most part, that manufacturer doesn't mandate what operating system you get to run on that PC. Certainly it doesn't get to tell you what software you can and cannot run on that PC - you purchase that separately and, similarly, whatever networking services is that you choose to consume, you get to choose where to pick them up".


14. GM: This was probably somebody grumpy about the iPhone, wasn't it?

PC: That's not the way the mobile phone has worked in the past. Typically, you buy your phone from the manufacturer, they tell you what software you get to run on it. You get to pay them for that software, you get to pay them for every byte that comes up and down the line and so on. There seems to be a general recognition that that world is coming to an end - that's a good way to put it. The operators are understandably a little nervous about this, but there seems to be a pretty broad consensus that, looking a little further down the road, a year or 2 or 3 from now, that's not the way it's going to be any more. We just got to adjust to it.


15. GM: Why does it make the operators nervous? I would think this would give them even more flexibility in producing products and services.

PC: Many of them seem to think that - how one of them put it - "We invested enormous amounts in money in building up these networks and providing the reliability. If we cannot recuperate that investment by implication, if we cannot recuperate it by charging people for the services that they are going to be consuming from us as opposed to having them go buy them from someone else, maybe we won't make the investment." But I think they are just going to have to.

Jul 27, 2009