Ruby On... SAP: One More Step In The Enterprise With A New Ruby VM
SAP, the market share leader in CRM & ERP and second largest business software company is adopting Ruby as part of their SAP NetWeaver and SAP ERP 6.0 solution. ABAP Virtual Machine will be able to run Ruby Code through the Blue Ruby extension.
In Timeless Software, SAPs CTO Vishal Sikka outlines SAP directions. Talking about software evolution, Vishal introduces the dynamics of change software industry is facing: business change, new technology layers, evolving infrastructure and finally emerging programming languages:
Even programming languages, and programming models around them, continuously evolve. Roughly every 10 years a major new language emerges, and minor ones emerge every 3 years or so, well within the lifecycle of large-scale applications. New programming models and robust developer communities can quickly emerge around new languages. The language Ruby, for example, is thought to have reached a million programmers faster than any other language ever.
Ruby on ABAP combines the best of both worlds as stated by Juergen Schmerder:
Lightweight, loosely-coupled, agile programming via Ruby, executed within the robust, proven SAP NetWeaver Application Server ABAP. In many ways, Blue Ruby makes the simple things simple and the complex things possible.
On the technical side, Blue Ruby is using parse tree to produce BRIL code (ABAP VM bytecode). Blue Ruby takes care of security using a bridge and a virtual file system:
- Secured bridge packages, which allow access to the functionality of the underlying host platform in a secure way by establishing a well defined sandbox concept.
At the end, you can run your Ruby code from within ABAP context (from Timeless Software):
We reuse the ABAP VM as much as possible for things such as garbage collection, session handling, memory management, etc. It should be kept in mind that unlike a normal Ruby environment, this is all running in the context of an application server – the NetWeaver Application Server (ABAP). This means that all of this is operating in the normal context of ABAP work-processes, roll areas, user authentication & authorizations, etc.
For the moment Blue Ruby covers 3365 out of 4910 totalRubySpec specifications (70.2%).
A Webinar is also available.
We will continue to follow next Blue phases which might include Blue PHP or Blue Python.
InfoQ had the chance to catch up with Juergen Schmerder, Blue Ruby PM at SAP and Charles Nutter from JRuby team. It was a good opportunity to have an interactive discussion around VM implementations.
InfoQ: Firstly how is it decided that such a big company like SAP will be supporting Ruby? Vishal Sikka CTO thoughts on programming languages? Client/developers feedbacks? Real need for lightness? An originally one-man side-project?
Juergen: Firstly, it is not decided that SAP will be supporting Ruby. We are experimenting with the language and our group is very positive about the usefulness of Ruby. But our mission is to prove that SAP should in fact support Ruby – and/or other languages.
The Blue Ruby project was kicked off mid of 2007 at SAP Research Center in Palo Alto (a group inside the Office of the CTO, managed by Ike Nassi). Since then we have implemented a reasonable subset of the Ruby language (in our opinion good enough to seek feedback) and put a lot of effort into a seamless two-way integration with our nearly 300 million lines of existing ABAP code. What we are doing is in perfect sync with what SAP’s CTO, Vishal Sikka, promotes in his vision around timeless software, and Vishal even mentions Blue Ruby in his blog as an example. The original main motivation for the project was the need for a light-weight environment for consumption and adaptation of our SAP application platform.
InfoQ: Are you the only developer in BlueRuby project? What is your experience with Ruby? With Ruby VM implementations? When did the project started?
Juergen: We have a small team working full time on Blue Ruby with a few developers in Palo Alto and two more developers in our SAP China Lab in Shanghai, and we are getting valuable contributions from other developers inside the company. Our Shanghai colleagues formerly worked on the Xruby project and have a very deep understanding of the Ruby internals and of languages, compilers, virtual machines and so forth in general. My personal experience with Ruby – well I got into the language when I joined the project in September 2007. Since then, I have pretty much fallen in love with the language and went through the MRI source code countless times. And just like in any relationship, the language still surprises me once in a while :-)
The bad thing about implementing a VM for the Ruby language is that I have to work more in the host language of the VM (in our case – ABAP) than in Ruby itself... Whenever I get to code some Ruby outside Blue Ruby, I am using MRI as a reference and JRuby for all the experiments we do on our Netweaver Java Stack. And whenever people at SAP ask me why we are limiting our efforts to the ABAP stack, my answer is that for Java we already have a perfect solution – JRuby...
InfoQ: How did you make your technical choice like using parse tree for parsing and building AST Ruby? No use of LLVM? Wasn't it possible to hack one of the existing VM like JRuby to produce BRIL code? You didn't want to be dependent on the Java VM?
Juergen: When we first started with Blue Ruby, we had our compiler implemented in Java using ANTLR for parsing. Later we switched to ruby_parser because we are aiming for a self-hosting environment. Blue Ruby should be able to compile its own compiler, so our compiler has to be written in Ruby. We are looking into LLVM in even more futuristic side tracks of our project, but as our target platform is ABAP, compiling Ruby to BRIL with a compiler written in Ruby was the most pragmatic solution. And our main point to prove was that you can run Ruby and ABAP code side by side on one call stack. That set the boundary conditions for some of our decisions (e.g. not considering Java)
InfoQ: Don't you think that the limitations about Ruby processes methods, IO operation and absence of threads might slow up the adoption of Ruby in ABAP?
Juergen: We won’t deny that the limitations you mention could deter Ruby purists from Blue Ruby. But our target developers work in the ABAP space anyhow. A developer with no relationship with SAP whatsoever would never touch Blue Ruby, because the added value of our implementation is specifically that is makes the Ruby language available on the ABAP stack. If you don’t care about the ABAP stack, you won’t care about Blue Ruby either... ABAP people, however, haven’t missed IO operations or threads for decades (well, they might have missed these functions but they were able to get along without it), so I don’t see these gaps being a showstopper. That doesn’t mean we are not working on closing the gaps over time, it just doesn’t concern us now. And from the feedback we’re gotten so far, I can say there’s people out there who love the idea of having Ruby running inside ABAP.
InfoQ: Charles was recently stating:
Listen, people: Ruby is hard to implement. Oh, it may look easy at a glance, and you can probably get 70, 80, or even 90% of the way pretty quickly. But there's some crazy stuff in that last 10% or 5% that totally blindsides you if you're not looking for it.
Juergen: Couldn’t agree more with Charles. Still, with time, JRuby has been able to get pretty close to the 100% number, so it’s not impossible, just difficult. In a research environment at SAP, we are not even sure if we will ever try to get these 100%. The future of the project relies heavily on the feedback we will get from the community – our main question is “is there demand for Ruby in the ABAP community”. So our next milestones will be driven by the requirements we get. Of course, there’s the no-brainers – e.g. we want to run Rails. But that’s still a bit down the road I guess.
InfoQ: What are the next features, developments in the pipe for next BlueRuby releases?
Juergen: We are continuously improving our core lib coverage and we are starting to bring the standard lib to Blue Ruby (at least the libraries that are implemented in pure Ruby, not C). And what I definitely want to see in the near future, is a Blue Gem. Besides that, we mainly went public with what we are doing to attract some early adopters from our SDN community, and we got a couple of developers interested. We will work with these partners, help them to build something useful (hopefully!) in Blue Ruby and let them drive our priorities. E.g. if there is more demand for a Ruby unit test framework for ABAP programs than for an SMTP lib, we’ll build a unit test framework... We are also trying to figure out ways to let others contribute code to our project – however, SAP does not have a tradition with open source, so we need to unlock some doors first.
InfoQ: Charles, you recently twittered about BlueRuby in those terms:
Ok, this is getting a little absurd...yet another Ruby impl, this time for SAP's ABAP VM
InfoQ: Could you clarify on this?
Charles: I certainly didn't mean BlueRuby was absurd. In fact, given the little I know about ABAP, it sounds like a great idea. The reason I tweeted that is because it seems like for every VM that can run Ruby, there's now at least one implementation (and sometimes more). I love that there's so much Ruby to go around, but it's almost absurd how many different implementations there are. I think it's a good thing for Ruby.
InfoQ: When you implement a Ruby VM today what's the biggest error you will not redo?
Charles: Well, I would never attempt to optimize Ruby before making it compatible. There's two major reasons for this. First, if you optimize and then work on compatibility, you're very likely to optimize in ways that break that compatibility. And second, once you already have things running really fast, it's very tempting not to fix those edge cases that slow things down. Ultimately any Ruby implementation that truly wants to be compatible needs to be compatible *first* and work on performance later, because it's a lot harder than it looks.
InfoQ: With your experience on Java VM, is Java VM the VM of choice to implement a new Ruby VM? Is Java the language of choice for such aswell?
Charles: I think the JVM is by far the best way to get a new language up and going quickly, whether it's static or dynamically-typed. It's certainly not perfect; we've had to do a lot of gymnastics to get JRuby performance as well as it does, and custom implementations like MacRuby will probably have better straight-line Ruby performance in the short term. But we get so much benefit from the JVM it outweighs the costs; JRuby's core class implementations are by far the fastest around, even faster than the raw C implementations, and that's largely because of all the extra optimization the JVM does for us.
As far as languages go...Java really isn't that bad a language. It's the "C" of the JVM, and it still represents the most straightforward mapping to JVM semantics of any language. There are many other languages that are better suited to writing applications on the JVM, like Scala, Ruby, Groovy, or Python, but Java is still king for writing bare-bones, fast-as-hell library and framework code. That's not going to change any time soon.
InfoQ: Could you tell us the advantage of using JRuby over parse tree or any other solutions, to exploit Ruby code? And how could you convince Juergen to use JRuby or JRubyParser?
Charles: Well I must confess some ignorance here... since I don't really know how ABAP works. The new JRubyParser project was spun off primarily so that external consumers of JRuby's parser--mostly IDE projects like NetBeans, RadRails, and 3rdRail--could collaborate on a codebase independent of JRuby's fast-moving dev process. We wanted to give those consumers freedom to build a better Ruby parser library for the JVM while also giving us the freedom to modify the parser and AST to support better performance and a smaller memory footprint in JRuby.
I suppose JRubyParser would only be useful to Juergen and BlueRuby if they can run Java alongside their ABAP code. But to be honest, I think their approach of using ruby_parser is an excellent one; I know Ryan Davis (creator of ruby_parser) put a lot of time and effort into the library, and I hope to use it for some projects of my own, since it has a nice API and very standardized output. It's probably not going to be as fast as a C or Java-based parser, but that may not be a concern for BlueRuby.
InfoQ: JRuby is quite a mature project, do you have an idea of the man years invested in it since beginning? What are the next features, important issues in the pipe? Are you still as motivated as you were on the first day?
Charles: JRuby is definitely mature. We've been production-ready for well over a year, and no other alternative implementation has reached that milestone yet. There are governments, telecom, utilities, and many others running JRuby applications. As far as man-years...I suppose it's hard to tell. All the JRuby committers have put in more than 8 hours a day for a long time, but really it's a labor of love. There's probably dozens and dozens of man-years in JRuby, and we're still moving forward very fast. Plus we're still seeing JRuby open up new opportunities for Rubyists, like this week's announcement of JRuby working on Google AppEngine. The JVM is most definitely the platform of choice for language and application development, and I think we're just seeing the beginning.
InfoQ: Charles, would you have any questions you would like to ask Juergen regarding his VM implementation that will make for good reading.
Charles: What features of Ruby have you found most difficult to implement so far?
Juergen: Well, that’s a difficult question. There are features we just don’t support at the moment (like network sockets, threads, C-libs and so forth) because there is no equivalent functionality exposed in the ABAP VM. We had to do a lot of iterations until we got the method dispatching an the exception handling right, but now we are pretty close. Most work is certainly the type system, and unfortunately it’s also the least interesting part. Going public will force us to make faster progress here – it’s not really difficult, it’s just that we are researchers and not really attracted by implementing String#fct2500
Charles: What are the Open-Source plans for BlueRuby? May community developers contribute to it?
Juergen: Personally, I hope that SAP will one day allow us to release Blue Ruby under an open source license and I am working towards this. However, SAP does not have a long history with releasing open source (we are beginners in that space just as we are in the Ruby community), and our lawyer have to find a way how we can mix our traditional ABAP application licenses with open source components – as Blue Ruby is quite meaningless without the ABAP apps. So currently, I am unable to make any promises. But we are just starting a trial program and got a number of SAP partners interested in participating. The results of this trial program will certainly drive our priorities and our partners will also be able to get more actively involved. As this is based on dedicated trial license agreements, it won’t scale to a larger community, but it’s a start...
Charles: What do you think are the strong points of ABAP with respect to running Ruby? What do you think are the weakest areas?
Juergen: Well there are a lot of good things to say about the ABAP environment. It is an enterprise application server by itself, so we don’t have to worry about scalability, deployment, user access management and a lot more. And the biggest strength of the overall application server are the existing applications – SAP covers every aspect (may it be fincancial accounting, HCM, CRM, SRM, SCM and all the areas I forgot to list) of business processes in any industry and nearly any country. With Blue Ruby, we get local access to these nearly 300 million lines of ABAP code that has been written. And we can consume, adapt and extend these application inside the system with a modern and efficient language.
On the negative side we experience every day that ABAP wasn’t as a host language for implementing other VMs, it was meant for writing business applications. And unlike Java, it is still meant for that purpose and has not evolved into a “C for the ABAP-VM” (to tweak Charles’ statement about Java). So we sometimes have to use unefficient ( = slow) implementation for compatibility reasons, because ABAP does not offer another way. ABAP is interpreted byte code with no JIT compilation, so we do in fact run an interpreter inside another interpreter. As we are constantly refactoring our code to fix compatibility issues or try better performing ways, we also experience that ABAP tools lack support for refactoring. We end up re-writing big portions from scratch, just because adjustments would take even more time.
However it wouldn’t be fair to say that these are ‘weak areas of ABAP’, because we are not using the language for what it was meant for. In a way we are abusing the langauge, so we certainly don’t complain about any shortcomings. Always keep in mind, we are doing research here :)
Ruby over ABAP - is this SAP's cloud PaaS play?
Shane Hastie on Distributed Agile Teams, Product Ownership and the Agile Manifesto Translation Program
Shane Hastie Apr 17, 2015