The Future of ParseTree

Posted by Werner Schuster on Apr 22, 2009

Sections

Process & Practices,

Architecture & Design,

Development

Topics

Ruby ,

Runtimes ,

Code Analysis

Tags

ParseTree ,

Ruby1.9 ,

JRuby ,

Static Analysis

ParseTree is a library that allows to access the parse tree of Ruby code - either Ruby source code or Ruby code already loaded in the interpreter.
Ryan Davis, who is one of the developers behind ParseTree, now makes it clear in a post on ParseTree's future:

ParseTree is dead on ruby 1.9 and there is no plan to make it work.

Because of changes to internals in 1.9, ParseTree simply can not work. I asked for hooks/options to allow us to get to the information but they never arrived.

Specifically if you're using ParseTree to access the AST of a live method/block/proc, you're SOL. If you're just using ParseTree to do static analysis, then you can switch to
ruby_parser in about a minute of work and you're good to go.

While ParseTree continues to work fine on Ruby 1.8.x, it won't work on Ruby 1.9.1. There are solutions for parts of ParseTree's functionality, though. Ruby 1.9.x comes with Ripper, which allows to parse Ruby code and analyze their parse trees.

ruby_parser is a Ruby parser written in Ruby (by Ryan Davis); the output is the same as ParseTree's output. Static Analysis tools that work on source are best served by switching to ruby_parser, mostly because it doesn't require code changes (ruby_parser's output is the same as ParseTree's). Ruby_parser also provides source locations of AST nodes, which is obviously necessary for code analysis or code transformation tools in Ruby. Using ruby_parser also allows these tools to be used across all Ruby implementations - ParseTree is only available for MRI (a mostly complete version for JRuby exists, but lacks some runtime features).

However, there is one feature that ruby_parser can't provide: return the parse tree of Ruby code loaded in an interpreter. Some libraries make use of this feature to analyze Ruby code at runtime. One of them is Merb's action arguments, which look at the formal arguments of a method. The feature a was already ported to JRuby without the use of ParseTree. Now, a port to Ruby 1.9.1 is available with the methopara project. Methopara came from work by YARV creator Sasada Koichi and members of asakusa.rb, one of Tokyo's Ruby user groups. The methopara project brings a new API to Ruby 1.9.1:

This gem is only for Ruby-1.9.1. Because Ruby18x have merb-action-args, and Ruby-1.9.2 and its successors will have built-in Method#parameters.

Do you use ParseTree? Can you switch to ruby_parser or is the lack of ParseTree on 1.9.x a problem?

1. Back to top

   How we replaced ParseTree in Ruby 1.9

   by Seamus Abshere

   You may be able to use the sourcify gem...
   
   

   # Ruby 1.8 with ParseTree gem
   proc.to_ruby
   # Ruby 1.9 with sourcify gem (may raise MultipleMatchingProcsPerLineError or NoMatchingProcError if you have nested procs)
   proc.to_source
   # Ruby 1.9 with sourcify gem (giving it a little help finding the right Proc)
   proc.to_source :attached_to => :name_of_proc_in_code

   
   
   Here's how my company replaced ParseTree in Ruby 1.9. I hope that helps!

   Reply

• Older >