BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Microsoft Surpasses Java's Dynamic Language Support?

by Werner Schuster on May 07, 2007 |
When .NET was first released in 2000/2001, the Java community considered it a "clone" of Java, both language and standard library. Comparing simple code samples surely support this impression. However, MS profited from many years of experience with Java, and managed to solve some issues that Sun only now realizes as problems. The impression that the .NET and the CLR are evolving faster than Java is not lost on the Java community.
Neil Bartlett:
I think it’s clear that Microsoft are innovating faster on the CLR. For example: LINQ is an extremely powerful new feature (based on Haskell monads, I might add); Generics were supported earlier and better in C# than in Java (both flavours were inspired by Haskell’s polymorphic type classes… hmm!); the CLR has better support for multiple languages than the JVM; and now it has the DLR,  which is probably two years ahead of the JVM being able to offer anything comparable.
Other  examples of this are modularization and versioning, which.NET solved by choosing the assembly, a collection of classes, as the basic deployment unit. Assemblies are equipped with metadata such as version information, unlike Java's Jar file which lack  versioning metadata. This is troublesome for increasingly large applications, which load many libraries. OSGi now provides a solution for this, Sun is busy adding something similar to Java 7.

The Java language keeps on catching up with C#, adding features such as Generics and features such as AutoBoxing,  Enumerated types or Annotations. C# now has anonymous expression support, which forms the underpinning of the LINQ technology. LINQ can be thought of a statically typed query language for many different types of data sources, such as XML, relational databases, but also arbitrary object graphs. The Java space, meanwhile, debates language minutiae such as language support for properties and which of four types of anonymous function to include in the language.   

With the release of the DLR, Microsoft has moved ahead again, this time in the area of support for  dynamic or scripting languages on the CLR. The Java space does not have a comparable effort going on right now. Miguel de Icaza, who started the Mono project, a clean room implementation of .NET, summarizes the  features of the DLR:

  • A shared type system for dynamic languages.
  •    
  • A shared AST that can be used by language developers to create new dynamic languages.
  •    
  • Helper/utility routines for compiler developers.
  •    
  • A common hosting interface, to embed the general purpose scripting language interface into your program, and allowing developers to extend applications with one or more dynamic languages.
  •    
  • Console support, they even have a simple console interface for doing interactive programming.

The shared type system is an important factor that will allow dynamic languages to interact and exchange objects. Jim Hugunin explains the rationale behind this and shows the situation in Java and how the DLR uses a different approach. Jim Hugunin should know the difference, after all, he developed the  Jython project, a Python version implemented on the JVM, only to later move to Microsoft and implement IronPython, a Python version on the CLR. Jim Hugunin explains one of the issues with this:

The wrapper approach can also have deeper problems. One challenge is just to figure out what object to pass.
For example, if Python has a PyString and it calls a C# function that expects an Object, should it pass the
PyString or should it unwrap it into a String? These kinds of subtle type issues never have a good answer.
Even worse are the nasty problems that can be caused by loss of object identity when objects are silently
wrapped and unwrapped behind the programmers back.  
These kinds of problems are definitely available in the Java space, for instance in the way JRuby 1.0 will handle passing Strings between Java and Ruby code:
  • Java strings passing into Ruby code will be encoded as UTF-8, with the implication that you should expect to be working with UTF-8 byte[] in the receiving code
  •  
  • Ruby strings passing out of Ruby into Java libraries will be assumed to be UTF-8, and the resulting string on the Java side of the call will reflect that assumption
  •  

The Java space doesn't really any of the mentioned items, except for the hosting interface, which was added in Java 6, under the name of JSR 223. This is basically just framework to add new language runtimes and initialize and access them in a standardized way.  

Jim Hugunin continues with a detailed explanation of how dynamic method dispatch is handled, which makes use of extension methods and other existing CLR systems.  The only comparable initiative is JSR 292, which among other things wants to add a new bytecode invokedynamic .This effort was started by Gilad Bracha, who soon after the creation of the JSR, left Sun, and is now not convinced that this project will bring any short term solutions:
JSR 292 is an initiative I started to address these problems. I hope it will continue in my absence, but it will be some years until it comes to fruition (if ever). It is frankly much harder to add support for these features to a JVM then to stabilize Strongtalk.
Note: Strongtalk is a Smalltalk implementation, whose VM is the base for the Hotspot technology, that has been shipped with Sun's JVM for a long time now.

Danny Coward, Specification Lead on JSR 292, feels more confident about performance gains:

Creators of dynamic language engines are in the business of taking, say, Ruby code and turning that into Java bytecode. When today's JRuby engine tries to convert that method call into bytecode, it has to create a synthetic interface to represent the return type. That isn't an interface a developer creates, but is one purely created by the JRuby engine [so] that it can take that method invocation and [turn that into] the bytecode. And that was just the return type—the same holds true for method parameters and exceptions.

JSR 292 removes the need for that synthetic interface. Today, dynamic language interpreters must output the methodinvoke bytecode, even when interpreting, say, a piece of Ruby code. Tomorrow, with JSR 292, they will be using the invokedynamic version. It will streamline the engine implementations because a lot of today's engines are worrying about creating new synthetic types and doing lots of book-keeping: when a method is called in seven or eight different places, they have to re-use that synthetic type all over the place.


It is important to note, that these changes will go into the JVM Specification, which means that they'll be hard coded and will not be easy to update in the future. A library based approach has the advantage that if a better way of handling these systems is found, it can be used immediately. The JVM based approach will remain the same for a long time, because JVMs tend to have a long shelf time (as a reference: Java 1.3 is still in use in companies).  It also remains to be seen whether the JVM can actually make use of this bytecode and really improve the speed of dynamic method invocations.

Another issue is the official support and backing of JVM based languages. Currently, JRuby has two of its developers on the Sun payroll. One of them, Charles O. Nutter, has started to reach out to communities such as Jython and Groovy, and it remains to be seen if these efforts will start.  Considering that MS has teams closely cooperating on IronPython, IronRuby, and the JavaScript and dynamic VB support, it has a certain edge here.After all, the DLR is a product of different teams sharing their experiences and factoring them out into a common library and knowledge base, whereas the JVM based language teams often have to relearn important lessons.

For instance, JRuby features a Just In Time (JIT) compiler, that will turn Ruby code into Java bytecode at runtime. The problem: in the current version, this code breaks the usual  set_trace_func based debuggers (which use a callback approach to implement debugger functionality), by not calling this callback. This means, that the debugging story of JRuby is impacted by this. This same issue must be handled and solved for every language, so sharing at least some of the experience and maybe code, will save time and work.

To add some perspective: Microsoft's DLR must still prove itself, at the moment it is available online together with IronPython. IronRuby hasn't been released yet, and it's real world speed and general interoperability remain to be seen. Java still has the benefit of being viewed as more open and is available on more platforms than .NET, although Miguel de Icaza seems convinced that Mono can ship  support for Silverlight by the end of the year.

May 10th update: Added a "?" to the end of the title to reflect the lack of consensus on this issue that came out in the discussion thread - Floyd.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Flame bait much? by Charles Nutter

Wow Werner, I kinda expected better from you, unless of course your goal was to invite flames from many communities. To say that the CLR is ahead of the JVM for dynamic language support ignores so many details...

- How many languages are implement for the JVM and for the CLR? How about multiple implementations of those languages? JVM wins hands down here.
- How much of DLR is finished and being used by many languages? Only a handful of languages are actually based on the DLR...fewer even than Microsoft's previous attempts to provide language-agnostic interop layers.
- When I told you the JRuby JIT didn't handle set_trace_func, I also told you it would be easy to add. I chose not to do it since most people weren't running with trace functions and JIT at the same time. It's like one line of code, man...you can't characterize the JVM or its bytecode as being some critical blocker here.
- IronRuby is probably one notch above vaporware here...and already it's introducing gross incompatibilities like require returning component objects and initialize being a callable method to set those components up. We're a few weeks into IronRuby and already they're doing things that will damage support for popular Ruby apps and libraries. That doesn't bode well.

DLR seems to be little more than IronPython's underlying libraries pulled out into a reusable form, which although admirable is certainly not the way I'd approach implementing a common runtime for dynamic languages. To begin with, it takes only one (or a few) language's perspective into consideration...and by most accounts, Ruby is not Python. Second, signs point toward it forcing languages into incompatible boxes from which they'll never escape. DLR is early research and headline-fodder at best right now. All due respect to the guys working on it...it's a damn hard problem to solve...but articles and headlines claiming CLR has suddenly surpassed JVM for language support are absurd.

And then there's the type interop of which you speak, and try to make JRuby an example again. JRuby supports a different string representation for a key reason: we feel that compatibility with the language and its libraries is more important than breaking the language to accommodate the limtations of the platform. Our approach may be wrong, but it's certainly captured the hearts and minds of many folks in the Ruby community. I'd wager that the largest reason why projects such as Jython haven't gained as much mindshare is because of continuing incompatibilities with the core implementation...incompatibilities that prevented key apps and libraries from working...incompatibilities which Microsoft seems to happily swallow and propagate to other languages through the DLR.

So unless your post was intended as a gross misrepresentation of the facts, I'd say you have some research to do.

Re: Flame bait much? by Dan Tines

Wow Werner, I kinda expected better from you, unless of course your goal was to invite flames from many communities. To say that the CLR is ahead of the JVM for dynamic language support ignores so many details...


What communities? Sun developers like you?


- How many languages are implement for the JVM and for the CLR? How about multiple implementations of those languages? JVM wins hands down here.


That's just plain wrong. There is a website that lists lots of so-called "languages", but in reality that list doesn't amount to much. What JVM languages are truly usable as of right now? Groovy, Scala, Nice (if you count a defunct project), JRuby, CAL (a Haskell like language). What else? It's pretty embarrassing, but not surprising, considering that Sun has always had the attitude that Java is the only language you need....well, until recently.

On the .NET side: C/C++, C#, VB, IronPython, Nemerle, Javascript, F#, Boo, various flavors of traditional languages like ObjectPascal, COBOL, etc...

There's no comparison. .NET wins hands down. Sun never understood enough to to be able to mentally divorce the Java language from the rest of the Java stack.




- How much of DLR is finished and being used by many languages? Only a handful of languages are actually based on the DLR...fewer even than Microsoft's previous attempts to provide language-agnostic interop layers.


As opposed to absolutely no equivalent in Java land?



- IronRuby is probably one notch above vaporware here...and already it's introducing gross incompatibilities like require returning component objects and initialize being a callable method to set those components up. We're a few weeks into IronRuby and already they're doing things that will damage support for popular Ruby apps and libraries. That doesn't bode well.


One step above vaporware is a wrong statement to make.




DLR seems to be little more than IronPython's underlying libraries pulled out into a reusable form, which although admirable is certainly not the way I'd approach implementing a common runtime for dynamic languages. To begin with, it takes only one (or a few) language's perspective into consideration...and by most accounts, Ruby is not Python. Second, signs point toward it forcing languages into incompatible boxes from which they'll never escape. DLR is early research and headline-fodder at best right now. All due respect to the guys working on it...it's a damn hard problem to solve...but articles and headlines claiming CLR has suddenly surpassed JVM for language support are absurd.


Why, because you're a Sun employee and can't handle the truth. I'll leave out the dynamic language support comparison for the moment, but the CLR is way ahead of Java on the number of usable languages that is has.




So unless your post was intended as a gross misrepresentation of the facts, I'd say you have some research to do.


Charles, it's you the one that needs to do some research here. I guess nobody is surprised that a Sun employee is going to be throwing out lies, but it's pretty embarrassing that Java has fallen so far behind than the CLR in so many areas.

Re: Flame bait much? by Michael Neale

OK some more detail:

JVM:
Groovy, Scala, JRuby, Nice, CAL (a Haskell like language), Javascript (Rhino), Python (Jython), Java, Scheme (Jscheme), Haskell (Jaskell), COBOL (PerCOBOL, commercial) ... I could go on...

hows .net sound now?

Re: Flame bait much? by Dan Tines

OK some more detail:

JVM:
Groovy, Scala, JRuby, Nice, CAL (a Haskell like language), Javascript (Rhino), Python (Jython), Java, Scheme (Jscheme), Haskell (Jaskell), COBOL (PerCOBOL, commercial) ... I could go on...

hows .net sound now?


A whole lot better than Java.

Re: Flame bait much? by Christoph Henrici



A whole lot better than Java.


How stupid. Who are you anyway? Boiling down enterprice computing issues to Sun vs. M$.

Re: Flame bait much? by Kieron Wilkinson

Heh, well done Dan. The little credibility you might have had with your first post, you just lost.

Perhaps you didn't realise that people are interested in balanced opinion, not fanboyism. Grow up.

Re: Flame bait much? by Dan Tines

Christoph, maybe you didn't get the memo. This isn't the serverside or javalobby. Who are you?

Re: Flame bait much? by Dan Tines

Kieron, you and the Java zealots lost all credibility a long time ago.

Re: Flame bait much? by Michael Neale

Great reply. Really, thanks for enhancing InfoQ.

My only grip with this "article" is the title, which is clearly done in that way to attract maximum attention for InfoQ, which I find disappointing and slightly cheap. Otherwise its an interesting read (Charles also has some gripes with the content I guess).

>What communities? Sun developers like you?
2 minutes of research will show Charles was working on JRuby (with many others) long before Sun took it on (now he has spare time ! yay !). Now JRuby is used fairly wideley, there are even commercial products built on it. I guess I share his gripe that a headline like this is really touting research ware (or vapourware if you want to by cynical about it) - which dilutes the value of the research, and also the work and expertise of people like Charles.

CLR is great, the DLR looks fascinating for sure, as an R&D effort. Its all good, and competition is good.

Re: Flame bait much? by Dan Tines

>What communities? Sun developers like you?
2 minutes of research will show Charles was working on JRuby (with many others) long before Sun took it on (now he has spare time ! yay !). Now JRuby is used fairly wideley, there are even commercial products built on it. I guess I share his gripe that a headline like this is really touting research ware (or vapourware if you want to by cynical about it) - which dilutes the value of the research, and also the work and expertise of people like Charles.


And what does that have to do with the original question of "What communities?". You mean InfoQ should watch out not to offend the JRuby "community" because the article says that Microsoft is ahead of "Java" on the dynamic language front?

Some corrections to the article by Crazy Photon

Assemblies are equipped with metadata such as version information, unlike Java's Jar file which lack versioning metadata.
That's not true, you can specify that in the manifest file.
With the release of the DLR, Microsoft has moved ahead again, this time in the area of support for dynamic or scripting languages on the CLR. The Java space does not have a comparable effort going on right now.
Yes, they are in the process of adding support for them in Java 7.
JVMs tend to have a long shelf time (as a reference: Java 1.3 is still in use in companies)
The same can be said for .NET (.NET 1.1 is still in use in many companies).
A library based approach has the advantage that if a better way of handling these systems is found, it can be used immediately.<quote></quote>Not immediately, you have to distribute the library and do the applicable code refactorings.

Re: Some corrections to the article by Tero Vaananen

The same can be said for .NET (.NET 1.1 is still in use in many companies).


I have no statistics about 1.1, but if I had to guess which version is the dominant .NET version in live deployments by far - I'd say 1.1. The same thing will happen with other .NET versions - very slow transition.

Transitioning from 1.1 is not actually plug and play. I don't know how bad it is from 2.0 to 3.0, but if it is anything like 1.1->2.0, we won't be doing it in the next 5 years, and who cares about the versions after that, 10 years maybe.

Re: Some corrections to the article by Jonathan Allen

.NET 3.0 and 3.5 just contain addition libraries for the 2.0 runtime, so there are no inherit transition costs.

As for 1.1 to 2.0, I'm still being unpleasantly surprised from time to time.

Jonathan Allen
InfoQ

What's up with the title? by Chris Nelson

This is a great article, but why post with a title that deliberately presents an obviously subjective opinion as fact? Floyd, where are you man? I expect better from you guys. Don't let InfoQ become another TSS, please!

Flame me all you want, BTW, but I won't be back to read further comments after posting.

Re: Flame bait much? by Werner Schuster

Hi Charles,

I don't think this article is a dig against, JRuby - after all, I use JRuby and do throw bug reports and - possible - features your way, so I'm certainly not biased against JRuby.

With that out of the way, your questions:
Number of languages on the JVM:
Yes, I know about that old list with 100+ languages,all of which have something to do with the JVM:
www.robert-tolksdorf.de/vmlanguages.html
Most of them are research toys and pretty much dead and not maintained anymore. The ones that are alive, can be grouped into Java preprocessors, or other Java derivates. There are a few dynamic languages, like Jython, in there. But: you also know how Jython has fallen back in it's support of the official Python - I think we can agree on that. There are folks working on it - as far as I can tell, but let's see what comes of that. And after all: go back two years and look at that list, find JRuby: think of JRuby 2 years back. Would you call that a useable tool? You obviously know how _much_ work went into making it into something that actually works like Matz' Ruby. So... with that eye, look at all the other languages in that list, who have much fewer developers, much less public interest, much worse PR and hence smaller communities. JRuby has now 2 (or three, I guess, as Ola works on JRuby at TW too) paid developers, and there's still much work to do.
What I'm - rambingly - saying: the list of JVM languages is long, the actually usable language list is shorter.

How many languages on the DLR:
4 languages (IronPython, IronRuby, VBx, JavaScript), at various levels of completion.
Counter question: how many languages use invokedynamic to improve their speed? - Nonsense question of course, as no one can even explain what the thing is supposed to do ... sure I know Gilad's slides and other hand waving... but no one has a concrete, detailed explanation how exactly it should solve all problems. DLR is here now, it has momentum (paid developers), will be developed openly (as it seems) and will gather experience with speeding up the implementations. invokedynamic is a mystery... and you say so yourself:
headius.blogspot.com/2007/01/invokedynamic-actu...
This is what I was trying to say here... Sun's a lot of talk, but MS actually _shows_ the code.

set_trace_func:
The fact that 1.0 is near, and the JIT generated code doesn't call it is a bug - I probably should have filed it. If I use a debugger, and happen to have the JIT turned on, I still want the debugger to work correctly.
The fact that you can add it easily is great, and if it happens before 1.0 the issue is settled. BUT: IronRuby already compiles down to CLR IL, and already has debugging information in it - despite the whole thing being only a few months old. This is another thing I was trying to say: JRuby, XRuby, Jython, Rhino, etc etc etc are different communities and they all have to do these things over and over. Remember the recent discussions about getting better debugging support in JRuby (jruby-debug)? IronRuby, as it does the same as all the DLR langs, already has this, mostly because it can use the libs from IronPython and company.

IronRuby:
I don't know how IronRuby is today, but John Lam keeps on stating that he intends to be compatible. Let the .NET Ruby community keep him honest... they won't like incompatibilities. The guys who work on the Google SoC RSpec projects (I have an upcoming interview with them here on InfoQ) will surely provide some helpful tools for shaming him into making IronRuby compatible - in case he or MS doesn't.
And please also remember: JRuby also has it's incompatibilities, like no Continuation support, or things like subtle differences in Regexes (which, I believe are solved for 1.0). I _know_ the reasons for them, and they're good reasons, but still: it behaves differently from Matz' Ruby. Same standards for everyone.

As for languages: they have different languages, even VB which, for instance, needs method dispatch to be case insensitive, etc. And all this is available in a library that's available and developed online. invokedynamic's behavior is not known yet, will be cast into the JVM/JVM Spec, etc, so Sun has exactly one shot at getting this right.
If you have confidence in that they get that right: fine, we'll see about that in a year or two. For now: invokedynamic is vaporware... actually it's even a vapor idea, so MS has an advantage as it has already shipped some code, as alpha or beta as it might be.

I give you the string argument; your (or the JRuby's team) decision was certainly the right way to go.

As for flames: this was not intended to ignite flaming, I actually spent a lot of time making it as neutral as I could.
If the Java community feels upset... well, I've been in the Java community for 10 years, and being upset is what Java community members do best. Whether it's a C programmer claiming that Java is slow, or MS releasing .NET, or DHH saying nasty things about Java web apps... there will always be people taking out the pitch forks and torches to riot.

Re: Some corrections to the article by Werner Schuster

@Crazy Photon:
About metadata in JAR files:
You're right, about the manifest file. But this sentence was trimmed to much and lost some of my argument. The version number in the JAR is absolutely worthless. If you have code that wants to use, say, JDOM 0.9, then it's nice if the JAR manifest contains the version number... it won't help you if JDOM 1.0 is also on the classpath and happens to be loaded earlier.
With .NET, you can specify which versions you want to use. In the Java space, one solution right now is OSGi, which allows you to have different versions of the same library in the same address space, and your bundle can specify the exact version it wants to use.

If you've ever worked on a large, extensible application (non-JEE), like Eclipse, you'll know that versioning like this is crucial.

Re: Flame bait much? by Ola Bini

So, I'm not going to go through your answer, just provide a few small comments. First, I agree with other commenters that the article, and especially the title felt very subjective and flamebaity. This has nothing to do with the Java community, it's about the style of your writing in this article.

Secondly, regarding incompatibilites: there is a huge difference between incompatibilities that are caused by the implementation difficulties (we _could_ provide Continuation support, but that would make JRuby worse in many ways, since we would need to get away from native threads. .NET can't possibly support this either). The compatibility issues we are fearing from IronRuby looks like they are on a totally different level, changes which have been introduced with no justifiable implementation problem being the cause. And, it's also a difference between features people use all the time (like require and the top level scope), and things which isn't widely used (like continuations, and very obscure parts of regular expressions).

Re: What's up with the title? by Michael Neale

yeah that is what worried me. Hopefully its just a glitch, and InfoQ can get back to quality content AND headlines.

Re: What's up with the title? by Obie Fernandez

For the record, the title was not without controversy internally at InfoQ, but we try to give authors a good amount of latitude. I personally would have preferred the title as a question.

Re: Flame bait much? by Kieron Wilkinson

Dan, I'm looking for balanced opinion. I am afraid that the only one coming off as a zealot here is you.

Re: Flame bait much? by peter lin

Microsoft's DLR is a step forward. How big is still unknown and no one will know for a while. There are several other languages i can think of that run on JVM.

LISP
Beanshell
jelly

Since I've written my own "LISP-like" interpreter, from my experience one need not add features to Java language or the JVM to support dynamic languages. Does it help? Yes, it helps, but it's an exageration to say adding things like invokeDynamic make a "huge" difference. It does make a difference if you want to generate bytecode, but I would argue why? If you can run interpreted and be just as fast or faster, the difference is purely a preference.

take JESS rule engine as an example. It supports CLIPS language, which is derived from LISP. JESS run wicked fast in interpreted mode without the support of invokeDynamic. my bias 2 bits

peter

Re: Flame bait much? by Dan Tines

Dan, I'm looking for balanced opinion. I am afraid that the only one coming off as a zealot here is you.


Yeah....right Kieron. Notice how many comments this story has compared to the rest of InfoQ articles? Wonder why? I'll tell you why. Java zealots have their panties in a bunch.

Re: What's up with the title? by Floyd Marinescu

I take full responsibility for the title. We normally never state a subjective opinion as a fact, and its unfortunate that we've descended into a thread somewhat similar to the other mentioned sites. ;)

The title was my idea because I understood that the DLR did represent unquestionably better support for dynamic languages than what's in the Java platform (that's Java, not JRuby itself). Perhaps from this thread that was not a correct conclusion, so I've added a "?" at the end.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

23 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT