InfoQ

News

Oniguruma Java port speeds up JRuby

Posted by Werner Schuster on Nov 28, 2007 10:30 AM

Community
Java,
Ruby
Topics
Ruby on Rails ,
Performance & Scalability ,
JRuby
Tags
JRuby ,
Language Features
Ola Bini reports that Joni, a port of Oniguruma, was merged into the JRuby trunk:
This is a glorious day! Joni (Marcin's incredible Java port of the Oniguruma regexp engine) has been merged to JRuby trunk. It seems to work really well right now.
JRuby team member Marcin Mielczynski took the job of porting the Oniguruma Regex engine to Java code - Oniguruma is the Regex engine included in Ruby 1.9.x.

This might just be the last installment in the (seemingly) never ending story about JRuby and Regular Expression (Regex) engines. Early JRuby versions used Java's built-in Regex library (included since Java 1.4) to implement Ruby's Regexes. While this was the simplest solution, not requiring any 3rd party libraries or ports, it also brought problems that made it unsuitable for JRuby. Since JRuby aims to be a compatible implementation of Ruby 1.8.x (or future versions), it's necessary to support the same Regexes. Java's implementation turned out to be incompatible, partially because of algorithm details that caused it to fail for some of the expressions. Ola explains the steps that followed:
To fix that, we integrated JRegex instead. That's the engine 1.0 was released with and is still the engine in use. It works fairly well, and is fast for a Java engine. But not fast enough. In particular, there is no support for searching for exact strings and failing fast, and the engine requires us to transform our byte[]-strings to char[] or String. Not exactly optimal. Another problem is that compatibility with MRI suffers, especially in the multi byte support.
All these problems seem to be - or will be - solved with Joni. Regex performance has been a big problem in the past (e.g. see Lessons from building Oracle Mix on JRuby on Rails), but Joni  seems to help with that too. Charles Nutter looked at REXML performance with the new code:
After running through a series of basic optimizations, most of the key expressions we worried about were performing as well as or much better than JRegex, so Ola went through with the conversion over the past couple days. Marcin is continuing to work on various optimizations, but both Ola and I have been playing with the new code. And it's looking great.
The linked article continues with the benchmark results comparing the code before and after the merge, which shows significant speed ups with the Joni code.

These issues also show a problem shared by many alternative Ruby implementations.  Rubinius, a Ruby implementation written in (mostly) Ruby, uses the simple solution of including Oniguruma. Ruby implementations based on VMs such as the JVM or .NET, however, have the problem that including a native library makes deployment more difficult (they'd need to ship platform specific versions). Not just that, as Marcin explains in a comment on Ola's blog, there are other integration issues:
We've been thinking about [including Oniguruma] already. There are few reasons:
Threading: Oniguruma uses global locks when initializing code range tables or managing shared AST nodes (like Character Class hashtable). Oniguruma bytecode interpreter also uses thread locks (it can be turned off but we get it for free in java land, and it'd be a hack to mix foreign threading with java one).
Exceptions: it would be hard to recover from segfaults. Converting Oniguruma errors to Ruby exceptions would also be an ugly hack.
JNI: it requires data separation, so all strings/bytes would have to be copied.
Additional binary distribution: good luck compiling it one Mainframe :D

No comments

Watch Thread Reply

Educational Content

Bindings, Platforms, and Innovation

This presentation focuses on the Internet and separating myth from fact, history from the future, and the mundane from the imaginative. Bob Frankston presents a vision of what could and should be.

Orchestrating Long Running Activities with JBoss / JBPM

This article explores the use of JBoss and jBPM to implement design solutions that effectively address the issue of orchestrating long running activities.

Neo4j - The Benefits of Graph Databases

This presentation covers the use of graph databases as an optimal solution for data that is difficult to fit in static tables, rapidly evolving data or data that has a lot of optional attributes.

Realistic about Risk: Software development with Real Options

This session introduces Real Options and shows how it can help in running your project. Real Options is a decision-making process that can be used to manage risk.

Communication Flexibility Using Bindings

This article discusses the use of bindings on services and references (including the instance of non-configured bindings) as the means to implement SCA communications in a Web and SOA environment.

Writing DSLs in Groovy

After a short introduction to DSLs, Scott Davis plays with the keyboard showing how to approach the creation of a DSL by typing working snippets of Groovy code that get executed.

Scaling Agile with C/ALM (Collaborative Application Lifecycle Management)

IBM Rational and InfoQ present, Scaling Agile with C/ALM, an eBook showing organizations how to become “finely tuned software delivery machines” by enabling team integration and scaling.

Concurrent Programming with Microsoft F#

Amanda Laucher presents a real life enterprise application written in F#. She shows actual code snippets, explaining design decisions and suggesting how to use some of the F# constructs.