1. Venkat, which JVM languages do you have experience with?
Well, Java predominantly of course has been around for a long time. I started the programming in Java right about the time it was introduced. Then of course as years went by I started picking up other languages and notably of course Groovy and I picked up Groovy a few years ago, I’m going to say maybe five or six years ago, I can’t remember exactly when I did. Programming with Groovy for a while. And I wrote a programming Groovy book along the way and then I got excited about Scala and picked up Scala along the way and then I wrote a programming Scala book. So I would say most notably those are the two languages I play with most of the time on the JVM. I also have worked a little bit with at the Clojure so I would say these are the four languages and of course a little bit JRuby as well. So I tried to stay up with most of these languages but definitely Scala and Groovy more in depth than the others.
The interesting thing is there is an overlap between some of these languages and certainly you know one language is a little bit better than the other in certain areas and often times I have people come and ask which language should I choose and I would say: "If I were to eat an ice cream I would probably pick a neopolitan where I have multiple flavors in it". But I would say a typical enterprise application is like that: typically when you’re developing an enterprise application you could have various things you want to do - you want to talk to the database, you want to perform some XML processing, talk to web services, do some numerical computations possibly, various other things you do and, you know, most of the times on a Java platform programmers have been using Java pretty much for everything and that’s like saying I’m going to have one type of vehicle to drive around this entire world.
What’s really nice about a language like Groovy is: if you ask me what really excites me in Groovy I would say that one thing is metaprogramming and with metaprogramming you’re able to take a program and extend it by adding capabilities at run-time and if you look at the DSLs (domain specific languages), a DSL really requires two things - one is (I’m talking about internal DSLs here) and you want the host language to be extremely flexible but also you want the language to be very concise and expressive, in other words you want this language to have as little as a ceremony as it can. Like semicolons, parentheses, you know so on. If you really only look on that point of view you would argue Scala is a great language because Scala doesn’t even require a dot in method calls, you know, in various contexts and clearly Groovy and JRuby are candidates to this as well.
But if I really want to develop an internal DSL with enormous amount of flexibility and extensibility I would argue, in addition to the low ceremony of the language I can imagine not having metaprogramming so I think if I want to build an internal DSL then Groovy is a much ideal language or JRuby would be an ideal language because they support metaprogramming and have very flexible and expressive syntax and a low ceremony. So metaprogramming and writing the layer of the code, that needs a flexible idea to evolve dynamically. Groovy is a great language for that. On the other hand if I really want a static type language where I want to express certain boundaries of my application interface I want to build, let’s say, an infrastructure.
If I’m going to not build a web application but I’m building a web server on application server or maybe I’m building a product in my company which runs on a server with a huge amount of performance need I would more tend towards a language like Scala because Scala is a more statically typed than Java itself, Scala has some really flexible XML processing capability, high conciseness, highly effective expression of code, a really nice concurrency API. Essentially well proven approaches in Erlang has been brought over to Scala, got an actor-based concurrency model. So I would say for those cases I would use Scala.
So if you were to kind of look at this as a stack below the infrastructure layer I would say something like Scala, top layers where I need more flexibility and, you know, freedom to really turn round and make it more extensible, more metaprogramming need in DSL layers, I would say probably lead towards a language like Groovy.
There are a few but there are always ways to get around it very easily. But let’s go back for a minute and talk about how would I probably use these languages in a typical application. In front of us we have two options: one option, maybe not so desirable, is to write some classes in Java, some classes of code in Groovy and some classes of code in Scala and then just toss them round and mix them in. Yes, we could do that but I would say that’s probably not most desirable. If you look at an application stack and an enterprise application you could have nice architectural boundaries and than you may want to have a certain language you use in a certain boundary and then the communication could happen using a JSON or XML or other formats or even may a database could be a way to communicate between them depending on the application.
So, if you’re going to take a second approach where these languages fill in a certain architectural layer than there’s no issue at all in communication between them because the communication happens using either JSON or XML or what have you. So you can develop a very light way of communication between those layers. If you were to use the first option which may be more suitable in some small selective applications there are two issues you’re going to run into: the first issue you’re going to run into is in the most simplest form I would say there are going to be certain key words in some of these languages that are considered to be not an idiomatic way or even not an acceptable type of syntax in another language.
Just an example: Scala uses colons in method names, Java is going to freak out if you have colons in method names, JRuby uses question in method names, Java is going to freak out when you have questions in method names. So if you’re going to call from Java into these languages for these method names you may run into an issue to begin with but Java 7 has the exotic identifiers who get around it. In Java 7 you could put a pound in a double quote and then you put method names and Java is going to be very forgiving accepting it. On the same token if you look at a language like Scala and Groovy there are certain keywords in these languages that it may find offensive when the method names or field names have those names in Java for example.
Let’s say you have a Java class with a method name yield, Scala is going to freak out. But the good news is all these languages give you an escape mechanism. So just like how Java is introducing the exotic identifier almost all language that comes second and later already planned for, because they expect this to happen and they have ways to escape from this collision. So you may put a back deck on and you may put a method name under the field name and it may allow you to escape through for example. So that’s a way to get around but a probably a more significant drawback or impediment is going to be when your Scala code or Groovy code is going to expose closures and you want to call those closures from within Java code. It’s going to be a little hard to do it because there is no real common closure mechanism.
It depends on how the language implements on the JVM and how you can tunnel through that. The simplest way to solve that particular problem would be if you do realize that you need to call into these layers from Java code itself than maybe you want to develop a more of a hygienic interface that is more comparable with the Java as a very thin surface layer or area and that let the code Java communicate through it and again I would keep this communication coarse-grain and then lot more fine-grained communication happen within the language boundary itself and that I think would solve some of the problems.
I wouldn’t at this point make this tight communication where in a half the methods are in one language and then another half are in another language and then I mix them together is certainly not a desirable. This could probably be an experiment somebody writes to see just how things work but if I am developing a serious enterprise application chances are I would lean towards more of an artificial boundary and then make the communication layer thoroughly small and then make that more of a, you know, database of communication if it all or a protocol based communication or if it’s going to be a language based communication then I would make it really small as well where it’s more compliant.
So it is an issue that could give us a hard time but there are ways to design a way and once the address said: "What it really boils down to is it helps me in my mind to think about these languages essentially as different dialects of Java." That just tells me that these are not just alien things dropped on us. Just like how you can do javac and compile Java code into byte code you could use scalac to compile Scala code into byte code, you know groovyc to compile Groovy code into byte code. And then once you are done it’s byte code talking to byte code and in my mind it becomes easier to say: hey for one here Spring what do I do. Well, you definitely add Spring jar into your class path and then you go about using the stuff from it.
In a similar way you take your code, add the class path to it, add the class path to either Groovy jar or a Scala jar and then off you go to use it as long as you can take up these few minor issues. So interoperability is not, you know, as smooth as a 100% but it’s probably just a small hurdle and knowing a few of these issues I think we are thoroughly in a good way to handle it.
That’s a very interesting question and in fact on one hand I would say "Here’s a platform that really was a single language multiple platform to begin with" even though people argue that really was not the intent for most part that’s how we perceived it. Unlike other friends in the .NET community where we have multiple languages and maybe a single platform to a certain extent if you want to argue that way, it’s been surprising and while we are looking at the way the platform has been able to support it and I would say partly the reason for that is these languages that have come up on the Java virtual machine, have found a thoroughly nice way to provide all these different constructs. If you could look at Groovy for example Groovy is a dynamic language with an optional typing on it to a great extent.
Scala itself is something that provides a hybrid style coding but even though these are different paradigms of programming they have found a tactical way to map into the constructs that the virtual machine and, you know, the bytecode so to say just in time compiler supports and so on. So I would say these languages have been thoroughly smart and tactical in mapping the constructs in the underline layers. Is it perfect? It’s really not in a number of ways, for example we don’t have clearly a closure in Java at this moment so these languages that do support closures are bending over the back and using anonymous inner classes and so on. So not a very ideal thing but from a programmer point of view I think they have done a thoroughly decent job in hiding that for most part.
Yes, you may a little run into certain issues in edge-cases, but for the most part of your programming in these languages you don’t really see the pain of those. So that’s something to these I would give credit to the language designers they have done for a decent job in mapping over to these underline constructs in the virtual machine. Can this be better? I think this can be better in a number of ways: certainly Java can accommodate closures and I would really remove the problem we talked about a little bit earlier in terms of calling closures into other languages, from other languages that would certainly eliminate. But there was one other biggest concern that was in the virtual machine, especially with respect to dynamic languages.
When you look at a language like Groovy you could pretty much call a method into an object not even knowing what this object really is and it turns out that whenever everything is said and done, at the byte code level that are really four calls and one of them is in invokevirtual as we know it, which allows you to call into the polymorphic methods. There’s in invokespecial which allows you to call into like private methods and so on and there is inovkestatic which allows you to call into static methods and there is inovkeinterface, which allows you to call into interface methods. And all of these four it turns out, are fairly statically typed. You really have to know what the type of the object on which they are invoking these methods and so what these dynamic languages have done is they have gone through the back door.
They go through their own implementation of call sites, they end up using a combination of indirect calls and, reflection, and so on and as a result what the consequence is when you have these four calls like in invokevirtual, invokestatic, and the two others I mentioned the JIT compiler gets a hand on these things. It’s fine to watch this. It’s a way to find a method in Java as a final method and then write another method as non-final and simply call these two methods, you know, compiled this byte code and have a look at the byte code, you know, using a jad or javap -c and so on you’d be surprised to find both the final method and the non-final method, both of them really are invokevirtual and the reason why they are invokevirtual as the compiler says "I don’t really have to work really hard to optimize this because right behind me the JIT compiler is going to come in and optimize it."
So the JIT compiler will take these to invokevirtual calls and based on the context and what’s loaded into virtual machine and so on, you can optionally decide to inline appropriate methods and then do other tricks with that. Well, the great news is when we have these four calls I mentioned you got really good optimization by the JIT compiler. Well, when the dynamic languages go through a back door, they pretty much shut the ability for the Java JIT compiler to perform this optimization. And as a result right in the beginning the dynamic languages have lost the advantage of performance optimization and other optimization that you can benefit. That language is like Java and Scala can benefit being statically typed languages.
Well, the good folks maintaining the virtual machine and the related work decided it’s time to fix it and so in Java 7 that’s where there are introducing the invokedynamic. And what invokedynamic does is it says: "Hey, dynamic languages when you want to make a call into an object where you don’t know what this object is at compile time, you don’t have to go through the back door. Go ahead and make a call directly through this invokedynamic and we will give an opportunity for the JIT compiler to perform the optimization that it can perform." So now I would say that capability can bring the language at the same leveling field as these other languages in terms of making use of the optimization that the compiler can provide.
I think that’s a real welcome news so improvements like that can be very useful, and of course it would be great to have closure implementation at that level as well but you know that’s probably going to have to wait a little longer for it but I’d see the invokedynamic capability as something really good. I think that can help us a great deal and that what’s even better is as a person programming on one of these dynamic languages you don’t have to do anything because you write the code like you normally do and rather than your code being compiled down into something that you go through a back door with a not so optimizable code your compiler that you use for your dynamic language would now compile it down to using invokedynamic.
I think this is a nice thing to have, a blessing for the programmers, is they don’t have to put any effort. The hard working people writing the compilers for these languages put the effort to modify this and then we can get a better performance for this. It’s not clear at this point how much of the performance is going to be but the chances are that I think we can do certainly better than what we do. This would also I think give an opportunity for this languages to, so to say, take time to clean up a little bit in their implementation of these dynamic methods. To be fair though, I love these languages but like everything else - this has been developed and developed over the years and we tend to accumulate technical debt taking along the way and this can be a great opportunity for them to reset and say: "All right, let’s go ahead and make use of all these constructs, eliminate these various levels of the processing we do and then call directly into the invokedynamic."
So, on one hand the JVM has been really capable already but the folks have been really smart enough to realize that it can be better than what it is rather than being complacent about it. And I’m really thrilled because finally there is a clear recognition that there are other languages than the JVM which may actually give us more productivity in certain areas and so it only make sense to open up for a better use of these languages and I think they are doing the right things going along in this direction. So I think it’s a pretty good time to be on the virtual machine using some of these languages. I am pretty excited for the future in that regard.
5. Which languages have the best support from a developer tooling perspective?
I would say none of them, if you really want to compare it with the capability of Java itself. You know, when I’m asked this question it brings back painful memory of 1996-1997 timeframe VisualCafe, don’t mention that but some of us wrote code. Of course that was the second IDE after Notepad for most of us and look at what happened! I mean I try forgetting some of the time frame. You know, over the years you don’t think about when things happen but I would say probably we really saw thoroughly decent IDE support around turn of the century. That’s when we really got better support for Java. So in a lot of ways, I would say, these languages are at the time. I’ve been looking for a language like Groovy for quite a few years and the IDE has come a very long way.
Compared to where they were before, how they were, if you compare to where Java is, I think we still have a long way to go and this is something that you know really concerns me a little bit and I would say the developer community is really split across three types of developers. There are a small set of developers who often work in a either a bleeding edge or a leading edge and they really don’t care about the IDEs. What they care about is the language strength and I’ve done you know production code in some of these languages including Ruby and honestly I would say I don’t really miss the IDE because I’m able to get around and write a code for these languages fairly quickly, the productivity gain they give for outweights of lack of IDE support.
But at the same time often times what you hear the people say which is rightfully so is that it’s very hard to do major refactoring in some of these tools also and maybe one answer is to do smaller refactorings along the way and not really get to the point where we need major refactoring. It also requires a certain slightly different mindset. The way your write code changes between these different languages. So the way you write code changes between these different languages. So the way you write code in Java would be quite different from the way you write code in JRuby or in Groovy for example. There is a lesser need for IDEs but that doesn’t eliminate the need for IDEs completely.
So I would say IntelliJ IDEA support has come a long way and I have a lot of hopes with IDEA, I’m a huge fan of IDEA which I have been using for several years and I think if there is a chance of somebody doing extremely well in these languages and they are really showing capabilities quite a bit there are some minor refactoring in these capabilities but I think that can be improved along the way certainly. So I have a lot of hopes in that area and the second, you know, set of developers are the developers that come a little bit later who still are open. They would rather have an IDE but they are still willing to flex a little bit and take, you know, a little bit of trouble and then probably a bigger community of developers. I would say a language has become mainstream when both set of people start adopting and I don’t think any of these languages can become mainstream without having a fairly decent IDE support.
So if you ask me what’s one the biggest challenges today for these languages to be adapted fairly well - I hate to admit this but certainly I would say a fairly good IDE support is very critical. So where is it right now? I would say it’s kind of coming along. There was a time when you could debug this code within the IDE-s, that’s gone, we are past that time but a lot more has to be done and there are some very smart people working in these areas and I have a lot of hope that they would have really some good tools available.
One of the first challenges is, I would say, for people who need to maintain this code. It really depends on the organization of the company and the team size. If you’re looking at a fairly large team and you know, sometimes you may have one or two programmers who are, you know, really the alpha programmers, alpha geeks go out. They write the code over the weekend and the rest of them have no clue what this is doing. This is a language they have not even probably heard of sometimes. That could be pretty challenging. But if the programmers have participated in developing these codes I do have to say, I would say I program in fairly comfortably in eight different languages I would think if I were to sit down and count.
There are a number of days I actually switch between three or four different languages for my own work and I do have to say that there is probably periods of the day when my mind was not switched properly and I would be sitting there and writing, maybe you know Java code and you know a Groovy which is not a bad thing because it would be accepted which is slightly a problem but there are times I would write Ruby code in Groovy or Java or I would write Scala code somewhere in the middle and it would take me a couple of minutes to figure out: oh, that was wrong syntax altogether and it’s painful when that happens. You need to have a little bit of an attitude where you say: ok, fine, I messed up but I’m going to recover from and move on. But it really depends on the comfort level and the confidence the developers have as well. But unless the developers are thoroughly experienced I wouldn’t throw them to two or three different languages on a given day. It really becomes more painful.
So, in that regards if there is a decent architectural boundary and if you have a group of people working in one boundary and then a group working in another that maybe a little bit easier with a very small number of people who are very competent and experienced going across the boundaries and these are the guys who don’t have a problem mixing languages. I would say that would be better. So there is a balancing act here and on one hand we could say: "Well, all my programmers need to be able to look at it, some are going to restrict this all into one language." I’m not going to say here that Java is that one language that’s painful.
I would argue any of the languages we have talked about are painful if we want to do everything in it. I would say: "Language A is better in this particular case and language B is better in this other particular case and you make a better leverage by mixing these together in a project." So on one hand I do want to mix these languages, on the other hand I don’t want to mix them so much that people lose track of where these are being mixed in and for what reason and it becomes just a nightmare to understand architecture and code base. So I think a good architecture is still very important and once we make the right decision and say: "All right, here is where we really want to process DSLs, so we going to write that component in Groovy and the small set of guys who are maintaining the DSL language can then spend their effort maintaining the code base in Groovy."
And here is where we are going to maintain concurrency and I think we should really do an actor-based concurrency. So let’s go ahead and use maybe Scala or maybe a library that provides actor-based concurrency. Ok, maybe there are two or three guys who maintain the code base are going to be comfortable maintaining that? The rest of the team can have a general awareness of it. They don’t have to be expert into this. So when they look at a piece of the code that they understand what it is but they may not actually maintain it on a daily basis. So it really depends on the team makeup. On one hand I am speaking a kind of both sides of my mouth here. On one hand, you know, in a true Agile environment I would argue that everybody should be able to maintain any piece of code.
And certainly we do that on projects that we work on but our teams are a lot smaller and we have very talented set of people who emerged and learned some of these languages. On the other hand it could be a larger team, but then in that case I would say you probably have a small set of people maintaining different sets of code base and that those people can then focus on that area so it shouldn’t be too tough. So I think it depends on the team. We need to balance that based on what their competence levels of comfort level is and when I say "competence level" it’s not an insult on these teams. You know, we all are competent in certain things and not so competent in other things until we are trained ourselves to doing it, it’s just recognizing that we are not good at it and then spend the time and effort to get better at it but it’s important to know where we are and we develop what is comfortable to us I think.
My good friend Mark Richards says there is a triangle and at the bottom of the triangle which is bigger and then right in the middle which is much smaller than that and the very top which is a very small triangle and he says that in this triangle the very top triangle which is a smallest triangle is what you know and then the middle area is what you know that you don’t know and the much larger area in the bottom is what you don’t know that you don’t know and in a lot of times what we should strive to do is to move things from the bottom to the middle one so at least we know we don’t know that and I think that’s very important. That’s again where a team is willing to know what they don’t know and then deciding whether it’s a right priority to go on to learn it or it should be just put back and if it really becomes important at least we know where to go back and look at it.
That’s a hard one and the reason it’s a hard one is lot of times people ask me: "Ok, I’m going to learn this. How much time do I need? And that’s a question I cannot answer because unless I know their strengths and capabilities and what they know it’s extremely difficult to say it." But what I can say though is that - and I share with you the experience I had - I was teaching a course on one of these languages and there were two guys about the same experience, same age in the room and some of these concepts that I was presenting was a little hard to digest for one of them whereas the other person took it fairly well and every few minutes he would say: "Oh let me compare this to this other language I know".
And he compared in contrast and said: "Ok, that’s how it’s different and that’s how it’s similar" and it made me realize that day that if a person has only programmed in one language a period of let’s say for ten years this person is going to have the hardest time learning a new language, especially if this language is very different. If the person has learned, spent the time learning, let’s say one language every three years for example then this person would require less time to take up a new language compared to the other person who has learned one language in past ten years. In other words the time you need to learn a language is inversely proportionally the number of languages you learned over the past ten years. I think as a matter of fact if I’m manager in charge of an enterprise what I would do is I would not just focus on getting these people trained, you know, in a short amount of time but give an opportunity for them to continue to learn irrespectively what they going to use right away and it is to keep their mental muscles active.
The more they look at a different API, a different way of designing, a different coding style, a different framework then what they do is the mental muscles are continuously trained and when they do have to pick up something very different from what they are doing they can quickly pick it up. I had a hard time learning new things and this was a time when I really didn’t learn a lot of new things. So when I have to pick up a new thing it took a lot of effort and I’ve changed that over the last you know few several years I would say. I had no use for Ruby and I was just excited there were people who were talking about it and I said "Let me go give it a try" and so I started learning Groovy, played with it for a few years, and suddenly I had a need to write a production system with it, jumped in and implemented it and got really excited about it.
And now that I have knowledge of what Groovy is and how different it is from you know the traditional languages that I have bee used to at the time like C++, Java and C# and so on and then I saw Groovy come up and I looked at Groovy and said: "Oh gosh, Groovy is Ruby, but runs on the JVM (at that time that was really JRuby) and it also has a lot of similarities with Java, it’s got a lot more semantical match with Java itself", nice match between Java semantics and the object modeling Groovy and it took me very little time to learn Groovy because all that I had to do to learn Groovy was not learn Groovy but apply the differentials. "Here’s how it’s different from Ruby and that’s it’s similar to Ruby".
I had no reason to learn Erlang, but I got excited about it, spent time learning Erlang, played with it for about a couple of years and then I saw Scala coming along the way, jumped in and started picking up Scala and now again I had to say "How is this language similar to and different from Erlang and how does it carry the Java platform various semantics in it. Once again, I didn’t take a whole lot of time to learn Scala, but I took the time to learn Erlang. So it’s where you spend the time learning along the way, then you can pick up something very different very quickly. Have I not spent the time learning Ruby, had I not spent the time learning Erlang, then I think my transition into Groovy would have been a lot longer, the transition into Scala would have been a lot longer.
I was able to pick up these languages not only very quickly, but then go off and write a book on it as well, as each of them as well, only because I was able to relate back to a lot of experience I had developed over the time and then I can wrap up very quickly and get better at it. From a corporate point of view and an enterprise point of view, I would be focusing on my team constantly learning something different. Then that way, when they have the opportunity, they will be able to pick up. One question that the managers would ask is (and I’ve read this somewhere and I thought this was wonderful) "What if I give them the opportunity to learn these things continuously and then what if they leave"
This little poster I read said "You are worried that you may train the people and they may leave, but what if you don’t train them and they stay with you?" Something very important to think about is that as a community we start developing over professionals and then people leave one organization to another. Then we have elevated the entire field with us. So it is something for us to think a little bit broader and also I think this is where the user community is very important. I would say, if you are a Java programmer, you really have to get out of the Java user group or group to Scala user group or Groovy user group or whatever the group is or form a polyglot user group. This could be within your own company, this could be in a small town you leave in, it could be a larger city and it really doesn’t matter how many people show up, what’s important is the frequency with which we meet.
I’m here at a conference, I’m speaking here, but the real selfish motive is I want to learn and when I speak to people, there is not a day when I listen to somebody and I don’t go away learning something better, something new. As a single person, even working in a company, I cannot possibly learn and know everything, but when I talk to somebody else, they have been in a different situation, they have faced a different set of problems. There are certain things that are common between us, there are certain things different, but then say certain things they’ve used or looked at that gives me a pointer look at a little bit more and learn. A lot of times I also come across programmers who say "My boss would never let me do it!" but I think that’s professionally responsible because it is our life, our professional career and we need to invest in it.
My recommendation which I try to do myself is I would spend maybe 20 minutes a day when I completely take something that I don’t work on at all and spend the time learning. I’ll be fiddling with something here, which I don’t have direct use, but I can’t tell you how many times those things that I did when I didn’t have direct use really became prime time in what I do years later. I would go to the user groups and spend time learning what others have to say. I used to work for a company years ago and I really wanted to spend some time learning and the environment didn’t provide it, but I realized it’s my life and I can’t spend 5 years and say "Gee, I really didn’t have a chance to pick up because I worked for this company!" That just doesn’t work.
So I would go in very early in the morning on Saturdays, when nobody is at work, so nobody disturbed me with anything and I would completely devote this time to do something that was independent of my product that I was building, but something that I really wanted to learn. In the process I built a little tool, a little something that would be a good motivation to build and I contributed that back to the company I worked in, but the net result of this was the company had a little thing that they could use, but the greater win for me was that I learned something I didn’t know before. Essentially, learning is easier when you learn all the time.
If we go to work and do exactly what the current job requires, I think we are putting ourselves into trouble. I say this quite a bit, this message is for me as much as for anybody else: it is not important to be employed, it’s important to be employable. I think the honor is on us to really spend the time learning continuously.