LLVM and Ruby Roundup: llvmruby, yarv2llvm and regexpllvm, Rubinius

Posted by Werner Schuster on Oct 18, 2008 03:19 PM

Community

Ruby

Topics

Runtimes ,

Performance & Scalability ,

Domain Specific Languages ,

Scripting

Tags

Ruby1.9 ,

Rubinius ,

Virtual Machines

LLVM contains the letters "VM", but it's actually a framework for building compiler backends. In short, it comes with an instruction set (immediate representation) and a backend which optimizes the code and generates native code for a host of platforms (x86, ARM, ...). One of the real world uses is in Apple's OpenGL JIT.

Llvmruby (llvmruby GitHub repository) is project by Tom Bagby. It provides Ruby bindings to LLVM. Llvmruby is well integrated into Ruby. For instance, this code adds an add instruction (codeblock is an instruction builder).

codeblock.bin_op(Instruction::Add, 41.llvm, 1.llvm)

Llvmruby opens Object and adds llvm to turn them into values LLVM can handle.

After following the instructions in the README (getting and compiling LLVM and compiling llvmruby), it's possible to start experimenting, eg. with irb. Here a sample session (with some snippets of code taken from the samples in the llvmruby project):

# Set up LLVM
>> require 'llvm'
=> true
>> include LLVM
=> Object
>> m = LLVM::Module.new("test_module")
=> ; ModuleID = 'test_module'
# Create a function type - the second argument is the method signature,
# ie. the return type and the types of the arguments
# In this case, there are no return values or arguments
>> type = Type::function(MACHINE_WORD, [])
=> #
>> f = m.get_or_insert_function("test", type)
=> 
declare i32 @test()
# Create an LLVMBuilder object which allows to call methods and
# generate instructions
>> builder = f.create_block.builder
=> #
# Create an Add instruction - note how it's possible to pass regular Ruby Fixnums
>> ret = builder.bin_op(Instruction::Add, 41.llvm, 1.llvm)
=> #
>> fcode = builder.return(ret)
=> #
>> ExecutionEngine.get(m);
=> true
# Finally: execute the generated code
>> ExecutionEngine.run_autoconvert(f)
=> 42

For more information on using LLVM, the LLVM Tutorial shows how to implement a JIT for a simple language and more.

Two bigger examples using llvmruby come from Miura Hidek:

• regexpllvm: turns regular expressions into LLVM code.
• yarv2llvm: takes Ruby 1.9's opcodes (the 1.9 VM was called YARV before it was made the official Ruby 1.9 VM) and translates them to LLVM. It follows in the footsteps of similar projects, such as _why's Unholy (translates Ruby 1.9 opcodes to Python VM opcodes) or HotRuby (a Javascript interpreter for Ruby 1.9 opcodes). Unlike these projects, llvmruby is tightly integrated with the Ruby 1.9 VM, which offers interesting possibilities. Regexpllvm shows one possible way of speeding up the execution of code, ie. compiling a DSL down to LLVM opcodes. Other systems use this approach, eg. Java's XSLT implementation compiles XSLT stylesheets to JVM bytecode
  Another possibility: compiling Ruby code hotspots. At the moment, if a piece of Ruby code turns out to be a performance bottleneck, the pragmatic solution is to rewrite it in C (the idealist's solution is waiting for the Ruby VM to get a performance boost). With llvmruby, another option is possible: compiling the Ruby code to LLVM, ie. using yarv2llvm as a booster. Whether this provides the necessary performance boost depends on the code, though.

Llvmruby allows to experiment with LLVM without having to touch any C++ and also allows to do so incrementally with irb. A useful property, eg. for contributors to Rubinius (the project plans to use LLVM) to prototype ideas. The Rubinius team has been busy rewriting the VM in C++ and cleaning up some internals. This work is currently happening on the CPP branch of Rubinius in the Git repository, which also contains LLVM as an external library. The C++ branch should soon turn into the master branch. According to some recent Twitters, Rubinius has regained the ability to run a lot of Ruby code with the new C++ VM. Eric Hodel twittered this week

Rubinius' new vm can now run the core specs without crashing

Evan Phoenix recently reported:

Huzzah! IRB works again under the new Rubinius C++ VM!

How would you use LLVM with Ruby?

1. Back to top

   libjit

   Oct 24, 2008 1:55 AM by Roger Pack

   There's also libjit + ruby 1.9 [http://rumble.withoutpenandink.com:4559/] Note that libjit is now LGPL 2.1 so that's nice. I'm not sure how the speed compares with LLVM. Could someone please port psyco to Ruby for me? -=R

   Reply

2. Back to top

   Re: libjit

   Oct 24, 2008 9:49 AM by Paul Brannan

   Ludicrous has been around for over a year now and has the ability to selectively JIT-compile methods on both 1.8 and 1.9 (which reduces risk of a bug in the compiler affecting the entire application). I don't know how the speed compares to an LLVM-based solution either, but I suspect that though LLVM should generate better code, the difference should be minimal, because the bottlenecks are in places like block invocation and method dispatch. Whichever compiler optimizes these better will perform better in real-world applications.

   Reply

• Older >