Ted Neward on Interop & Office Integration

1. Ted, tell us what's possible with Java and .NET working together.

As a general rule I guess the short answer would be "all things are possible". But I think that a lot of the interest is going to lie in trying to take advantage of each platform's relative strengths. At least amongst the general community, anecdotally speaking, .NET's strengths appear to be in the client's space when talking about Windows Presentation Foundation, about the fact that Windows is installed on 90+% of all the PCS out there. Entreprise Java, not just J2EE containers but Spring and all that, are installed on a good number of the backends of the world, so the canonical suggestion would be "How do I make use of Microsoft/.NET related technologies on a client to talk to Java technologies on the server?" We don't have to stop at that point but that's where most people will be most interested in.

Things like using Office as a rich client platform, being able to create an invoice in Excel that knows how to call out to a J2EE backend to verify inventory, to verify part numbers, to verify price, while letting Excel be the front end as designed by Excel power users. Things like using Word to do the ubiquitous mail merge where the name and addresses from which to pull for the merge come from a J2 backend; SmartTags, the little drop-down things that show up in Office, Excel, Outlook etc being able to communicate to a J2EE backend. One of the areas that is of particular interest to me is Windows Workflow. In terms of how Microsoft is positioning a product there's really nothing corresponding to it in the Java space, in terms of trying to allow power users to design their own workflow without having to understand specific technical details like XML and so forth. Running workflow inside of Spring or Spring inside of workflow, depending upon your particular scenario. There's a tremendous amount of potential in terms of using these two platforms together. It's just a question of what makes the most sense for a particular company's needs.

2. What are some more use-cases that you see possible with MS Office and what are the options for integrating Office and Java?

In terms of all the possible scenarios for Office, I think that list can easily go for days, if not for weeks. Microsoft developers, particularly those who come from a more VB centric background, have known for years about the notion of using Office as a client platform. This is the Visual Basic for applications language space and in a lot of ways there's a tremendous amount of stuff that you can imagine Office doing. In the past September issue of MSDN Magazine it's all dedicated to Office development. The first article on that series lists all of the different possible ways you can integrate Office from one perspective using Office to save as XML which can then be sent or read by Java applications, to some of what I've described earlier, the notion of an Office front-end, a Word plugin, an Excel plugin, communicating across-process to a Java back-end.

Office has specific support for what they call research web-services ( the idea is the you can type in something and hit a key stroke and it will execute a web-service call to pull up information on this.) The original goal for that was for e.g. if you want it to have online thesaurus, online dictionary, those sorts of things regarding a particular word. But there's no reason we can't subvert that paradigm and make that into a look-up around a particular topic or element gets more information about if this is a letter that's written inside of word according to a particular template document. "Dear..." put someone's name there and hit F1 and make a call out to your Java base CRM, pull out the guy's information including his boss, his boss's boss, or info about his kids, all that's stuff that sales people love to track so that they can least pretend they care about you as a person (My apologies to sales people out there).

Those kinds of things that try to make pull out to these sales history info; "we've notice that you've purchased product x a year ago and we wanted a follow up on how your evaluation is going etc". Yet you're authoring in an environment where your end user, your power user has familiarity with. You don't have to rebuild the client side infrastructure again.

Even if you're the die-hard Java fan you can still do this even if you don't want to bring up Microsoft Word directly, because the Eclipse rich client platform (RCP) has support for, because SWT knows how to reach down to the operating system in this case Windows). The RCP has the ability to bring up a Word or Excel automation view through COM (in this case), rather than .NET, so that now I can actually have the Eclipse show running and bring that up and actually writing in Word so I can get the full support of the spell-check and thesaurus and grammar etc. and we'll still be able to do the F1 call out to the research service and get that info back and so on and so forth.

There's a shell game that we can play there. I don't want them launching Word because that looks odd to my users, I want them launching my Eclipse RCP program which then can host Word, which can then talk to your Java program.

To a lot of people from the Microsoft space I just launch Word; that's not an unusual thing. I load the appropriate document template and then I hit F1 and get the research pane.

In the case of Excel, I got a friend in the .NET space, Rocky Lotkha who VB developers will recognize. He talks about a particular company where he did some consulting with, where literally their whole manufacturing estimation process was run off of an Excel spreadsheet. It's like function point analysis or software developers, but this was in the manufacturing space. So he meets with the client, the client says "We want to run 100,000 of these widgets ", sales guys says "Ok what's the complexity of the shot, what's the amount of material required per shot"; he starts plugging in numbers into various points, so there's basically this really large formula that they've codified inside the Excel spreadsheet so he can run these numbers and have a printed document that he can give to the client and say "Ok this is our estimate", which typically includes a certain percentage mark-up for profit. Generally speaking manufacturing firms are at least somewhat upfront about that. He would hand that to the client telling him that's the written estimate. Traditionally the problem with those things has been that they don't have any awareness of what is actually going on in the manufacturing floor because they are disconnected. If you wanted your Excel spreadsheet to be connected in some fashion, you'd have to write some kind of Excel plugin that talks to that space.

Prior to .NET /Java there was never an easy way to do the server communication stuff. But now in the Java back-end you've got Java connector APIs which makes it very easy to talk to lots of different things including embedded systems and you've got .NET which makes it fairly easy to talk to Java backend either through web service call, or in this particular case since we know it's a .NET front-end and we know it's a Java back-end, there are commercial tool kits we can take advantage of. There's Borland Janeva JNBridge, JANET; these are all commercial tool kits that you can pick up. There are some open source projects. There's Remoting.CORBA, IIOP.NET; all these tools basically enable specific .NET to Java, Java to .NET cross-process communication.

3. In terms of these different interoperability you mentioned, how would I know when to use which, aren't web services the answer to everything?

I think it's fair to day that anytime someone tries to sell you "one size fits all" solution, he's probably trying to sell you something and web services have kind of been solved with that kind of reputation. If you ever want to talk across technology domains, you want to do web services. That's a gross over-simplification of the case. Web services are great for doing platform neutral interoperability across processes. I've got Java out here, I've got Python, Ruby, .NET , I've got any particular language you can name. The idea behind web services was to create basically a platform agnostic interoperability layer. That's why we go through XML because it was the widely recognized ubiquitous language for doing data. There's a variety of different channels over-which we can do this. Http is the most common, in some cases is over-used/un-used doing it over ftp for example, doing it over smtp and pop3, doing it across raw tcp-ip, doing it to even database rows, through JMS, through MSMQ etc. lots of different ways we can distribute these XML messages but generally speaking we do this because we want to do it out of process, we want to do it across processes.

In some cases the simplest interoperability scenario is to say: "look, we'll do it through a file". For example in the Office scenario Word and Excel both know how to save in an xml format; so I save that to an xml format and then I use Java or any other language to parse that xml and extract the salient parts into your program. You can also do this interoperability in process. There are two particular things I'll mention by name. One is IKVM, an open source project, now part of Mono. IKVM basically creates a JVM in .NET. It can be used in two ways. It can be used to take Java byte code and compile it quote on quote an turn it into a .NET assembly or it can do byte code interpretation. What it actually does is it takes the byte-code in the Java methods and so forth and turns it into CIL which then gets compiled into x86. It is not running Sun JVM it's kind of running a zone that purports to have all the same functionality as the Sun JVM.

The other is a commercial product called Jugger.NET from CodeMesh; they will actually load the Sun JVM in-process alongside the .NET CLR; because at the end of the day these are both just DLLs living inside of a process. You load the Sun JVM next to the CLR and he generates some .NET proxies to make it easy to call into the Sun JVM. About a year and a half ago MSDN asked myself and another speaker whom .NET developers would recognize but Java developers won't, her name's Michelle Bustamante and together we concoted a Java.NET interoperability sample where we had a ASP.NET page, we were gathering information off a form and then posting it with a BEA JMS queue using Jugger.NET proxies to do the actual interaction with the JMS queue. So I took the JMS client jar, generated the code mesh proxies for it and so literally inside of ASP.NET we've loaded the Sun JVM, called into it which in turn had done the JMS communication necessary to talk to the BEA server to post this message into the queue.

There is a variety, a whole continuum here that we can examine right. We talk about in-process which is probably the most complicated of the scenarios, simply because now you've got two virtual machines which are running side by side and you have to be careful about how much transition you do back and forth between them. This complexity is getting smaller because various companies are looking for ways to try to reduce that complexity; for example JNBridge now has a shared memory option, where instead of talking over a TCP/IP port now you're talking through a shared memory channel; it's all in the same process, so it's much faster. But you're still marshalling data, so if you have a complex object graph you have to serialize all that into one long byte array stream, push that through the shared memory space and then pull it back and deserialize it. As opposd to if it's a proxy based scenario that's designed for in-process use; we can look at certain isomorphic parameters, like an integer: if it's a 32 bit int, it's a 32 bit int, and hopefully we don't have to do any translation there. The in-proc space is definitely what I would consider to be the most complicated the out of proc space; you know you have web services over here which I would be considered the most complex arm, and then you work down to something like Object RPC, like the tool kits I mentioned , we ‘re trying to follow the standard rules of distributed objects and so forth. Down to the messaging space where I just take something in .NET and put it in some place for Java to pick up be that a JMS queue, be that a MSMQ, be that Tibco Rendesvour, etc. In some cases be that just a file on a file system. Then down to what I consider to be the simplest forms of which is to say resource interoperability which is "look we just treat the original resource as they are originally designed to be seen"; the most common is: .NET puts a row into a database, Java consumes that row out of the database and we're all done. .NET has ADO.NET, Java has JDBC we use the database and the schema in the interoperability format and we're done, there's nothing more to talk about. You've got this whole continuom of complexity. My general suggestion to people is to start here at the simple end and go to the right until you get to a point where it solves your needs and then you stop.

4. What about the choice between web services and in-process? What are the main factors that will lean you in one direction or the other based on your requirements?

One area that out-of-proc is going to hurt you is that if you're talking over the wire you're making a significant trip there. That's a significant performance hit. Even if it's on the local machine, just talking to another process, making an out-o-process call versus an in-process call can be four orders of magnitude more expensive. If it's to another machine on the network it can be even more expensive than that (sometimes five orders of magnitude). That's clearly a huge performance hit.

If I'm looking for example to host a Swing component or a Swing panel or something similar, inside of a .NET WinForms application or inside of a Windows Presentation Foundation application, that's huge. Just to spawn another Java process, open a pipe and start communicating back and forth across that pipe that's a horrendous performance hit. The in-proc objects are in the same process. Frequently you want to communicate out of process because of isolation barriers, because of centralization requirements, because you already have the Java code running someplace else and there's no point in pulling it all in. Those are issues that generally speak to the in-proc nature.

Part of the things you have to be concerned within in-process is that if you're handing objects back and forth between these two environments that means that both of them have to pin the objects in place so you're going to create some havoc with the garbage collector too. Obviously if .NET is handing a pointer to an object out of the CLR its garbage collector needs to make sure that the pointer does not go anywhere. So it has to pint it and similar for the Java side. If they both get pinned .NET called to Java who then called back to .NET for whatever reason, now they're both pinned and the GC is less efficient about its job.

Generally in the in-proc scenario you still want to have some fairly good component lines drawn, some good discipline on that sense. I want to make sure that when I cross that boundary I don't do it more often than I have to, so as to avoid some of those issues. Threading can be another issue. Am I on a .NET thread, am I on a Java thread, am I on the same operating system thread as I was before? This can play havoc with some thread local storage scenarios. You have to be careful about those things when you're talking in-proc. As a general rule, IKVM and CodeMesh make it a lot easier to do the in-proc stuff than two or three years ago when you're option was .NET calling into Java meant you wrote managed C++ DLLs that did JNI into the Java space. And Java you were talking about to a managed C++ DLLs that called into the .NET space calling. That's ugly. You can still do that but it's ugly.