RubyConf'08 Videos: Ruby VMs: Internals of YARV, Rubinius, MagLev

Posted by Werner Schuster on Dec 04, 2008

Sections

Development,

Architecture & Design

Topics

Compilers ,

Performance & Scalability ,

Technology ,

Runtimes ,

Dynamic Languages ,

Ruby

Tags

IronRuby ,

Rubinius ,

MagLev ,

JRuby ,

Ruby1.9 ,

RubySpec ,

RubyConf

RubyConf '08 featured many talks about the various Ruby VMs. The talks varied from in-depth technical coverage of implementation details, to tech demos and general talks about Ruby performance.

Koichi Sasada's talk "Future of Ruby VM", where he explains his view of the C implementation of Ruby as the pragmatic solution for the near future, despite some of it's problems such as the conservative GC.
He continues to discuss some optimizations which were implemented for 1.9, which are off for now, but should be enabled in 1.9.2. Examples of these optimizations are tail call optimization, stack caching, efficient method caching and a more efficient Fiber implementation.
At around 20:00, he mentions some of of his research interests. One of them is Ricsin, which allows to embed C source inside Ruby code. The sample source from the presentation slides should make the concept clear:

def open_fd(path)
 fd = _C_(%q[
    /* C code */
    return INT2FIX(open(RSTRING_PTR(path), O_RDONLY));
 ]) 
 raise 'open error' if fd == -1
 yield fd
 ensure
  raise 'close error' if -1 == _C_(%q[
     /* C Code */
     return INT2FIX(close(FIX2INT(fd)));
  ])
end

It's differs from RubyInline by allowing to put C snippets inside a Ruby method. The current version received special support in YARV. Ricsin's SVN repository is available publically.

Further projects are a Ruby to C compiler (at round 28:40), followed by Atomic-Ruby, which tries to shrink Ruby by allowing to exclude some parts.

At 41:00 the status of the Ruby MVM project is explained as well (InfoQ reported about Ruby MVM).

Evan Phoenix' talk on Rubinius explained the status of the Rubinius project and of its C++ VM.
The reasons for the rewrite of the old C VM to the C++ VM is explained. The reasons are type safety and how C++ helped make the code simpler and removed many manual checks.

At 18:04 the current state of the implementation of primitives is discussed (how to write primitives in C). At 26:30, method dispatch and the strategies used to make it fast are discussed. At 35:00, the current state of MethodContext ("stack frames") allocation is explained. In older version of Rubinius, these were heap allocated, but the current version tries to allocate them on a stack (in a dedicated memory region) to reduce allocation cost. However, MethodContexts can be kept alive beyond the lifetime of the methods that they were created for (eg. by a Closure). These objects are kept alive - but since the memory region dedicated to MethodContexts is limited in size, it will fill up at some point, which will trigger a GC run to clean up MethodContexts which are not referenced anymore.

At 38:40 follows a discussion of Rubinius' support for Ruby extensions, which is mportant to run C extension gems such as hpricot, mongrel, mysql, sqlite drivers. Rubinius' handling of the problems of extensions is also explained (eg. problems with the generational GC or segfaulting C code).

Another talk about these topics is "How Ruby can be fast" by Glenn Vanderburg. While not a VM implementer, he discusses some reasons why Ruby is or can be slow. At 07:00 an explanation of Garbage Collection gives an overview of the theory of Generational GCs, followed by an explanation of performance optimizations for method dispatch at 19:35.

"Ruby Persistence in MagLev" by Bob Walker, Allan Ottis, explained the current status of MagLev. Bob Walker explains the basic benefits of Gemstone's persistence model, which allows to simply persist object graphs instead of having to map them to a relational model, thus avoiding a whole lot of issues with ORM libraries and tools.

At 14:30 Allan Ottis follows with a detailed explanations of the inner workings of MagLev, starting with the implementation of the object persistence. At 18:50 he continues with an overview of similarities and differences between Smalltalk and Ruby and what problems they pose for implementing Ruby on Smalltalk. At 22:29 the compilation process is explained, as well as the current parsing solution (ie. a Ruby server to parse Ruby code and use the ParseTree gem to return an AST formatted as ParseTree s-exprs).

At 25:00 the execution modes, both interpretation and native code generation are explained. At 30:30 the implementation of the allocation of Contexts ("stack frames") is shown, followed by the details of the object memory (heap) and the garbage collector.

The questions start at 44:20, with questions about distributed caching, future parsing strategies (using ruby_parser), the behavior of transactions across the distributed object memory.
 

Some of the more demo oriented Ruby VM talks were

• JRuby: What, Why, How...Try It Now by Charles Nutter and Tom Enebo on the status of JRuby, Ruby 1.9 support, FFI and more.
• "MacRuby: Ruby for your Mac" by Laurent Sansonetti.
• John Lam's talk on IronRuby

Finally, this is rounded off by Brian Ford's talk on RubySpec, where he explains the RubySpec project which now has tens of thousands of tests which define Ruby's behavior - a crucial tool of every alternative Ruby implementation.

• Older >