Rubinius: Inside the Bytecode Compiler and Foreign Function Interface

| by Werner Schuster Follow 6 Followers on Oct 17, 2007. Estimated reading time: 1 minute |
The first of two recent articles about Rubinius is by Giles Bowkett, who tries to get started with Rubinius compiler development. The Rubinius compiler works by traversing Ruby Abstract Syntax Tree (AST), a tree representation of the Ruby source, using ParseTree s-expressions. This means it is an array using symbols to describe the data. An addition, for instance, would look like this:
[:call, [:lit, 1], :+, [:array, [:lit, 1]]] 

A literal looks like this:
[:lit, 42] 

To traverse the AST, ParseTree comes with the SexpProcessor library, which facilitates the creation of visitors. To analyze all node types of a Ruby AST, a subclass of SexpProcessor with process_XXX methods is created, where XXX is the name of the node. For instance, this handles the :alias node:
def process_alias(node)
 cur = node.shift
 nw = node.shift
# ...

The Ruby to Rubinius bytecode compiler is built in this way. For instance, a Ruby alias call is parsed into [:alias, :old_name, :new_name], which the compiler handles as such:
 def process_alias(x)
 cur = x.shift
 nw = x.shift
 add "push :#{cur}"
 add "push :#{nw}"
 add "push self"
 add "send alias_method 2"

The compiler takes the old name (in curr) and the new name (in nw), and creates the bytecode instructions (as strings) necessary to implement the functionality, which are then turned into the binary bytecodes executed by the Rubinius interpreter.

Having the compiler in Ruby makes it easy to get insight into the inner workings and modify it for experiments. Useful scenarios could include instrumentation of the generated code or a low overhead way of collecting statistics about the compiled code.

To look at the Rubinius source code, either refer to InfoQ's article about getting started with Rubinius development or just take a peek at the Rubinius source code online, for instance the current version of the Rubinius bytecode compiler.

The compiler is not the only aspect necessary for Rubinius. A complete standard library is necessary too. Marcus Crafter, of Red Artisan, provides a tutorial on how to add library functionality to Rubinius. The tutorial shows to use the Rubinius foreign function interface (ffi) to access native library calls. This is used to implement some missing library functionality, in this tutorial,  the POSIX call link.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you