Matrix Your Rails Functional Tests

Ryan Davis added a new way to test multiple edge cases in his latest test suite release: ZenTest 3.5.0. The way he suggests to make it clearer is by using a matrix. Let's imagine you have rights properties to test in your application (with orthogonal states: readable vs unreadable, ...). You would basically end with the following 4 methods:

def test_edit_user_readable
    some_setup_to_initialize_user_readable_context
    some_action_here_edit
    some_assertion_error_read
end

def test_edit_user_writable
    some_setup_to_initialize_user_writable_context
    some_action_here_edit
    some_assertion_edit
end

def test_view_user_readable
    some_setup_to_initialize_user_readable_context
    some_action_here_view
    some_other_assertion_view
end

def test_view_user_writable
    some_setup_to_initialize_user_writable_context
    some_action_here_view
    some_other_assertion_view
end

You can easily see that some pieces of this code (setups, actions) can be factored. But Ryan went a step further by reorganizing it into a matrix where the column headers represent the different setup cases, the row headers represent the action applied and the intersection defines the expected result for a given action and setup context.

The 4 test cases will look like this:

     
         
             
setups
             
 
             
:user_readable,
             
:user_writable
         
         
             
matrix
             
:edit,
             
:error_read,
             
:edit
         
         
             
matrix
             
:view,
             
:view,
             
:view
         
     
 
 

ZenTest will stores off the edge case through the setups method and the matrix method will create a test method for every applicable result:

def test_#{action}_#{setup}
    matrix_setup_configuration #{setup}.split(//) # global setup
    matrix_setup_#{action} #{setup}, #{expected}  # action setup + execution
    matrix_test_#{expected}, #{setup}             # expected verification
end

So at the end the setup configuration will be factored at one place taking into parameters the different edge cases, the actions and assertions will also be put apart.

Ryan Davis gives a clear visual example of test cases before and after matrixization:

Before	After

Click the images to see the code in more detail.

The test matrix model is another demonstration of the DRY (Don't Repeat Yourself) process philosophy. It also easily makes it possible for non-developer to read and add tests by only modifying the matrix.