BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News DocTest 1.0 For Ruby Released

DocTest 1.0 For Ruby Released

This item in japanese

Bookmarks
Included in the Python standard library, various DocTest Ruby implementations were made available starting one year ago by Tom Locke, Roger Pack, and more recently Dr Nic.
DocTest allows the easy generation of tests based on output from the standard interpreter shell (IRB for Ruby), cut and pasted into docstrings.

Copy/paste your IRB testing output as comment for your function:
# doctest: Add 5 and 5 to get 10
# >> five_and_five
# => 10
def five_and_five
5 + 5
end
Later you'll be able to run the test:
$ rubydoctest five.rb
=== Testing 'five.rb'...
OK | Add 5 and 5 to get 10
1 comparisons, 1 doctests, 0 failures, 0 errors
This will run with latest sources from the git repository. An updated gem will soon be available

Such docstring-driven testing has its pros:
  • Those who are familiar with IRB live testing will find it useful and natural to use.
  • One place to look at for the test.
  • Simple.
... And cons:
  • Large docstring should be avoided and put into a separate test file.
  • Some complex testing assertion are not available.
  • It can be tedious to test large outputs.
Duane Johnson merged his changes on Tom Locke's repository to make version 1.0 available.

A screencast demonstrating DocTest usage is available.

InfoQ caught up with Duane Johnson to discuss DocTest and docstring-driven tests.

InfoQ: Firstly, what led you to write DocTest? I read you looked at 2 previous implementations.

Duane Johnson: I am currently working on an improvement to a large ruby project (memorypress.com) that has very little testing and documentation.  In order to make this a clean upgrade, we will definitely need some kind of testing in place to make sure we don't break things unintentionally.  The trouble is, most of my time "testing" is spent at the irb console, because it's not always clear what things do and how I should go about testing them.  In other words, testing to me is a kind of exploratory thing--I need to see how things work at the same time that I formally define how things work.  Ruby DocTest is a perfect combination since I can copy and paste my successful experiments into the codebase and check off the tests as complete.

The latest code can be found on github? It's a merge with a previous version of DocTest? What did you add?

Yes, that's the latest code.  Earlier this week, I forked Tom Locke's project at github.com/tablatom/rubydoctest, and then merged my changes into his project once he gave permission.  I found Tom's code to be very readable, and his approach to comparing results by evaling (rather than direct string testing, as in Python's doctests) was compelling.  This allows ruby's unordered hashes to be compared successfully, for example.  I also took Roger Pack's idea of putting doctests into comments right in the ruby source file.  I didn't borrow any of Roger's code (at code.google.com) but I borrowed his ideas.  Now, Ruby DocTest gives the option of both the "inline doctest" style of Roger's package, with the "separate documentation in a markdown file" style of Tom's package.

Aren't you satisfied with all the Ruby test frameworks?

Yes and no.  I still like rspec quite a bit.  But for my own workflow, I always find that I'm trying to catch up with the actual code written because I'm too lazy to switch to another file (or create a file) and write the tests.  Often, I have this little niggling feeling that I should test some edge case of a method I'm working on, but I forego it because so many other things are on my mind at the time.  If I could just put an irb session in there, I thought, that would cover 90% of my needs.  So I did.

DocTest is outside the BDD cycle?

Yes, that's correct.  This is not a test-first approach.  My current needs, as mentioned above, are in documenting an already-existing project that is not well-tested.

Is DocTest supposed to be a replacement for the other unit test framework or just an additional tool?

To be honest, I don't know yet.  It's a new way of writing tests for me, so I'll be playing with it to find the answer soon.  I imagine my 90% target above is close... there will probably be some places where it makes more sense to rely on traditional rspec or Test::Unit cases.

Isn't the way DocTest annotates code polluting the reading?

I imagine the migration path of doctest to go something like this:

1. You write some code, and then add an inline doctest comment or two.
2. You make some other code modifications, and later come back to the original code.
3. You add more test cases, but the number of tests (or length) gets unwieldy, so you move them into a separate *.doctest file.
4. Eventually, you approach a fairly complete set of markdown files with code examples as documentation, with instructions and comments all around them.

One of the interesting side-effects of writing tests side-by-side with code is that it tends to encourage you to make each ruby source file independent--in other words, you want to make it possible to "eval" the file without any dependency errors, since that's what the doctests require.  I find that once I unshackle a file from its complex network of hidden assumptions (as in a Rails file) then it becomes easier to test, and I'm more aware of what each file is relying on.

Rate this Article

Adoption
Style

BT