News

Google SoC Series: Creating RSpec specs for Ruby runtimes

Posted by Werner Schuster on May 14, 2007 03:00 PM

Community: Ruby
Topics: Unit Testing ,; Dynamic Languages ,; JRuby
Tags: Rubinius ,; Google Summer of Code ,; RSpec ,; IronRuby ,; Documentation ,; JRuby

A detailed specification is necessary for all implementers of Ruby runtimes. Currently, to make a Ruby runtime compatible to the official Ruby provided by Yukihiro Matsumoto, the implementers have to check the C code to see what it does or, if that's not possible, simply ask Matz or the Ruby community how something is supposed to work. With more and more Ruby versions around (JRuby, Rubinius, MS IronRuby, etc.), there's a need for a formal specification. Even better if the specification is actually executable code instead of pages of prose. Two Google SoC projects aim to fix this problem by creating RSpec specifications for Ruby. RSpec is described as:

RSpec is a framework that provides programmers with a Domain Specific Language to describe the behavior of Ruby code with readable, executable examples that guide you in the design process and serve well as both documentation and tests.

This resembles Test Driven Development (TDD), where Unit tests are supposed to be written before the code. RSpec is actually focused on describing the Behavior of the code, and tries to make these description as readable as possible. Pedro Del Gallego works on the project "Cover the core of Ruby with RSpec", while Florian Gross works on the project "RSpec suite for Ruby implementations". When asked why there are two projects with such similar focus, Pedro explains:

Florian and I wrote this two proposal in parallel without knowing each other. it's a bit symptomatic that two GSoC projects are quite similar. Creating a central test repository its a hot topic in the Ruby scene. Sasada Koichi or Evan Phoenix among others wrote about it in different interviews. I think our project is a good opportunity to create a central test repository. We are going to collaborate with Charles Nutter, who is hosting two interesting project, RubySpec and RubyTest. Florian will primarily focus on language specs and me on library specs, but that shouldn't be seen as a strict separation. We'll try to make these as complete and detailed as possible.

Note: Sasada Koichi is behind the project known as YARV, a new Ruby VM that has recently been merged into the Ruby 1.9 branch. Evan Phoenix is behind the Rubinius project, a Ruby implementation written mostly in Ruby.

When asked why RSpec was chosen for this effort, Pedro explains:

That's an interesting question, Rspec has a big drawback, its not trivial for the implementors to run this framework over their VM. We don't want to reinventing the wheel, rubinius has a pretty mature test suite based in rspec and has some mechanism to run this test against other ruby implementation, also rspec its the most expressive and understandable test framework for ruby. I understand test not only like a developer tool to avoid bugs, they are also a good source of "living document".

Creating a spec for a system like Ruby is not trivial and a lot of work. Pedro details how he and Florian plan to tackle this, and where the challenging parts lie:

We'll try to concentrate on core infrastructure, the core language specs and the core library specs in ruby 1.8.6. It depends on how well we are doing. The priority its to create a good quality spec, try to avoid holes in corner cases. I think, this its a long term project, we can't fit all this point in the SOC schedule. Right now we are defining what its important and what our priorities are. Trying to avoid holes in a specification is always a difficult problem in a software system, even more when the system its language, where the the space of possibilities and corners cases are big. Then I think the challenge its to do good analysis that cover as complete and detailed as possible. And here its where the role of my mentor (John Hornbeck) will be important.

Creating the specification for Ruby is a big task, and a helping hand is always useful. Pedro points to the project's Wiki as one starting point, as well as the RubySpec and RubyTest projects. Pedro's blog is also a good source of information.