InfoQ

News

Oniguruma Java port speeds up JRuby

Posted by Werner Schuster on Nov 28, 2007 10:30 AM

Community
Java,
Ruby
Topics
JRuby,
Performance & Scalability,
Ruby on Rails
Tags
JRuby,
Language Features
Ola Bini reports that Joni, a port of Oniguruma, was merged into the JRuby trunk:
This is a glorious day! Joni (Marcin's incredible Java port of the Oniguruma regexp engine) has been merged to JRuby trunk. It seems to work really well right now.
JRuby team member Marcin Mielczynski took the job of porting the Oniguruma Regex engine to Java code - Oniguruma is the Regex engine included in Ruby 1.9.x.

This might just be the last installment in the (seemingly) never ending story about JRuby and Regular Expression (Regex) engines. Early JRuby versions used Java's built-in Regex library (included since Java 1.4) to implement Ruby's Regexes. While this was the simplest solution, not requiring any 3rd party libraries or ports, it also brought problems that made it unsuitable for JRuby. Since JRuby aims to be a compatible implementation of Ruby 1.8.x (or future versions), it's necessary to support the same Regexes. Java's implementation turned out to be incompatible, partially because of algorithm details that caused it to fail for some of the expressions. Ola explains the steps that followed:
To fix that, we integrated JRegex instead. That's the engine 1.0 was released with and is still the engine in use. It works fairly well, and is fast for a Java engine. But not fast enough. In particular, there is no support for searching for exact strings and failing fast, and the engine requires us to transform our byte[]-strings to char[] or String. Not exactly optimal. Another problem is that compatibility with MRI suffers, especially in the multi byte support.
All these problems seem to be - or will be - solved with Joni. Regex performance has been a big problem in the past (e.g. see Lessons from building Oracle Mix on JRuby on Rails), but Joni  seems to help with that too. Charles Nutter looked at REXML performance with the new code:
After running through a series of basic optimizations, most of the key expressions we worried about were performing as well as or much better than JRegex, so Ola went through with the conversion over the past couple days. Marcin is continuing to work on various optimizations, but both Ola and I have been playing with the new code. And it's looking great.
The linked article continues with the benchmark results comparing the code before and after the merge, which shows significant speed ups with the Joni code.

These issues also show a problem shared by many alternative Ruby implementations.  Rubinius, a Ruby implementation written in (mostly) Ruby, uses the simple solution of including Oniguruma. Ruby implementations based on VMs such as the JVM or .NET, however, have the problem that including a native library makes deployment more difficult (they'd need to ship platform specific versions). Not just that, as Marcin explains in a comment on Ola's blog, there are other integration issues:
We've been thinking about [including Oniguruma] already. There are few reasons:
Threading: Oniguruma uses global locks when initializing code range tables or managing shared AST nodes (like Character Class hashtable). Oniguruma bytecode interpreter also uses thread locks (it can be turned off but we get it for free in java land, and it'd be a hack to mix foreign threading with java one).
Exceptions: it would be hard to recover from segfaults. Converting Oniguruma errors to Ruby exceptions would also be an ugly hack.
JNI: it requires data separation, so all strings/bytes would have to be copied.
Additional binary distribution: good luck compiling it one Mainframe :D

No comments

Reply

Exclusive Content

Concurrency: Past and Present

Brian Goetz discusses the difficulties of creating multithreaded programs correctly, incorrect synchronization, race conditions, deadlock, STM, concurrency, alternatives to threads, Erlang, Scala.

ActionScript 3 for Java Programmers

Often the hardest part of changing technologies is language syntax differences. This new article provides Java developers with a transition guide to Actionscript which forms the foundation of Flex.

Neal Ford On Programming Languages and Platforms

Neal Ford talks about having multiple languages running on one of the two major platforms: Java and .NET. He also presents the advantages offered by Ruby compared to static languages like Java or C#.

Future Directions for Agile

David Anderson talks about the history of Agile, the current status of it and his vision for the future. The role of Agile consists in finding ways to implement its principles.

Nick Sieger on JRuby

Nick Sieger talks about the future of JRuby, Java Integration, and his work on JEE deployment tools for Ruby on Rails like Warbler.

Rustan Leino and Mike Barnett on Spec#

Rustan Leino and Mike Barnett of Microsoft Research discuss the technology in Spec# and its futures.

10 Ways to Screw Up with Scrum and XP

Henrik Kniberg talks about 10 possible reasons to fail while doing Scrum and XP. Maybe the team does not have a definition of what Done means to them, or they don't know what their velocity is.

Tips from a Top Sports Team Coach

This article outlines 9 principles Marc Lammers discovered while building the world’s best field hockey team, mapping them to software development practices.