Refactoring Automated Functional Test Scripts with iTest2

Introduction

Automated test scripts are known difficult to maintain. With wide adoption of agile methodologies in enterprise software projects, one of its core practices: automated functional testing proves its value, at the same time provides challenges to projects. Traditional record-n-playback testing tools may help creating a set of test scripts quickly, but often end up unmaintainable. The reason: application changes.

In programming world, 'refactoring' (a process improves software internal structure without changing its behaviour) has become a highly frequent used word among programmers. In simple words, programmers make code more readable and design more flexible during refactoring process. Experienced agile project managers allocate certain time for programmers to perform code refactoring or make it as part of process finishing user stories. Most of Integrated Development Environments (IDEs) come with support for various refactorings.

Testers who develop or maintain automated test scripts usually do not have that kind of luxury, but share the same needs to make automated test scripts readable and maintainable. It is difficult (the more test scripts, the more difficult it becomes) to get the test scripts back on track for releases with new features, bug fixes and software changes.

Test Refactoring

The objective and procedures of functional test refactoring is the same with code refactoring, but it has special characteristics:

• Target Audience
  The end users of testing tool are testers, business analysts or even customers. The fact is that testers, business analysts and customers generally do not possess programming skills, and this changes the whole paradigm.
• Script Syntax
  Code refactoring is mostly supported on compiled languages such as Java and C#. Functional test scripts, however, may be in a form of XML, proprietary vendor scripts, compiled languages or script languages (such as Ruby). Depending on the test framework, the use of refactoring varies.
• Refactorings specific to functional testing
  While some common code refactorings, such as 'Rename', apply to functional test scripts, they are ones are specific for testing purposes, such as 'Move the scripts to run each test case".

iTest2 IDE

A new functional testing tool: iTest2 IDE is designed for testers to develop and maintain automated test scripts with ease. iTest2 is written from ground up dedicated for web test automation, the test framework it supports is rWebUnit (an open source extension of popular Watir - Web App Testing in Ruby) in RSpec syntax.

The philosophy of iTest2 is: easy and simple. Trials showed testers without programming experiences could write their first automated test scripts averagely less than 10 minutes under mentoring. With iTest2, testers can develop, maintain and verify test scripts against functional requirements; developers can verify the feature is working on; business analysts/customers view test execution (in a real browser: IE or Firefox) to verify function requirements.

The test scripts created by iTest2 can be executed from command line and integrated with continuous build servers.

Walk through

An example worth thousands of words. We will walk through complete steps from creating two cases to making them readable and maintainable by using refactoring tools in iTest2.

Test Plan

For our exercise, we develop typical yet simple web test scripts for Mercury's NewTour web site.

Create test case 001

1. Create a project

Firstly, we create an iTest2 project, specify the site URL, and a sample test script file will be created as below:

load File.dirname(__FILE__) + '/test_helper.rb'

test_suite "TODO" do
  include TestHelper

  before(:all) do
    open_browser "http://newtours.demoaut.com"
  end

  test "your test case name" do
    # add your test scripts here
  end
end

2. Use iTest2Recorder to record test scripts for Test Case 001

We use iTest2 Recorder, a Firefox add-on records user operations in Firefox browser into executable test scripts.

enter_text("userName", "agileway")
enter_text("password", "agileway")
click_button_with_image("btn_signin.gif")
click_radio_option("tripType", "oneway")
select_option("fromPort", "New York")
select_option("toPort", "Sydney")
click_button_with_image("continue.gif")
assert_text_present("New York to Sydney")

3. Paste recorded test script in a test script file, and run it.

# ...
test "[001] one way trip" do
  enter_text("userName", "agileway")
  enter_text("password", "agileway")
  click_button_with_image("btn_signin.gif")
  click_radio_option("tripType", "oneway")
  select_option("fromPort", "New York")
  select_option("toPort", "Sydney")
  click_button_with_image("continue.gif")
  assert_text_present("New York to Sydney")
end

Now run the test case (right click and select 'Run [001] one way trip'), it passed!

Refactor to use page objects

The above test scripts work and rWebUnit syntax is quite readable. Some might question the needs for refactoring, and what is 'using pages'?

First of all, test scripts in current format are not easy to maintain. Let's say we now have hundreds of automated test scripts, new released software changed user authentication to use customer's email as username to login, which in turn means we need to change to use 'email' instead of 'userName' in test scripts. Performing search and replace on hundreds of files does not sound like a good solution. Also project members like to speak common vocabulary within the project, which has a fancy name: Domain Specific Language (DSL). It is nice to see them used in test scripts as well.

It can be done using page objects. A page in our context represents a web page logically, it contains operations provided to end user on that page. For example, the home page in our example has three operations: 'enter user name', 'enter password' and 'click login button'. 'Refactor to use pages' is a process to extract operations into specific page objects, and it is made quite easy to do so with refactoring support in iTest2.

1. Extract to HomePage

The login function is on the home page, and we will make it so. As user login is a well understood function, we make 3 lines of statements (enter username, password and clicking login button) into one operation. Select those 3 lines, then click 'Extract Page ...' under 'Refactoring' menu (keyboard shortcut: Ctrl+Alt+G).

Figure 1. 'Refactor' menu - 'Extract Page...'

This opens a window like below to for you to enter page name and function name, we enter 'HomePage' and 'login' respectively.

Figure 2. 'Extract Page' dialog box

The selected statements (3 lines) are now replaced with

home_page = expect_page HomePage
home_page.login

A new file 'pages\home_page.rb' is created with the following content: class HomePage < RWebUnit::AbstractWebPage

  class HomePage < RWebUnit::AbstractWebPage

    def initialize(browser)
      super(browser, "") # TODO: add identity text (in quotes)
    end

    def login
      enter_text("userName", "agileway")
      enter_text("password", "agileway")
      click_button_with_image("btn_signin.gif")
    end
  end

Run the test case again, it shall still pass.

Note: As Martin Fowler pointed out: the rhythm of refactoring: test, small change, test, small change, test, small change. It is that rhythm that allows refactoring to move quickly and safely.

2. Extract to SelectFlightPage

After login successfully, customers land at flight selection page. Different from login page, every operation here is more likely to be updated independently by developers, so we extract each operation to a new function. Move caret to the line

click_radio_option("tripType", "oneway")

Perform another 'Extract to Page..." refactoring (Ctrl+Alt+G), enter "SelectFlightPage" and "select_trip_oneway" for new page and function name.

select_flight_page = expect_page SelectFlightPage
select_flight_page.select_trip_oneway

3. Continue extract more operations into SelectFlightPage

Continue performing refactorings for the remaining operations to 'SelectFlightPage'': 'select_from_new_york', 'select_to_sydney', and 'click_continue'.

test "[1] one way trip" do
  home_page = expect_page HomePage
  home_page.login
  select_flight_page = expect_page SelectFlightPage
  select_flight_page.select_trip_oneway
  select_flight_page.select_from_new_york
  select_flight_page.select_to_sydney
  select_flight_page.click_continue
  assert_text_present("New York to Sydney")
end

As always, we run the test case again.

Write test case 002

Since we now have two pages ('HomePage' and 'SelectFlightPage') from refactoring test case 001, writing test case 002 will be a lot easier (by reusing them).

1. Using existing HomePage

iTest2 IDE has built-in support for page objects, typing "ep" and pressing 'Tab' key (called 'snippets') will expand to 'expect_page' and populate all known pages for selection.

Figure 3. Auto-complete pages

We get

expect_page HomePage

To use HomePage, we need to get a handle to it (in programming world, it is called 'variable'). Perform "Introduce Page Variable" refactoring (Ctrl+Alt+V) to create the variable.

Figure 4. 'Refactor' menu - 'Introduce Page Variable'

home_page = expect_page HomePage

Now type "homepage." in next statement, the functions defined in the page class will show up for you to choose.

Figure 5. Page function lookup

2. Add dedicated operation for Test Case 2

Test Case 002 is quite similar to Test Case 001, the differences are trip type selection and assertions. With the help of the recorder, we can identify the new operation:

click_radio_option("tripType", "roundtrip")

Then refactor it into a new function in SelectFlightPage

select_flight_page.select_trip_round

Here it is

test "[2] round trip" do
  home_page = expect_page HomePage
  home_page.login
  select_flight_page = expect_page SelectFlightPage
  select_flight_page.select_trip_round
  select_flight_page.select_from_new_york
  select_flight_page.select_to_sydney
  select_flight_page.click_continue
  assert_text_present("New York to Sydney")
  assert_text_present("Sydney to New York")
end

Run the test scripts for Test Case 2 (Right click any line in test case 2, and select 'Run ...'), it passed!

Reset application to initial state

But wait, we are not quite finished yet. Test Case 1 passed, Test Case 2 passed, running them together got an error on Test Case 2, why?

We did not reset the web application back to initial state, the user remains signed in after finishing execution of Test Case 001. To make tests independent from each other, we make sure the test execution starting with sign-in and end with sign-off.

test "[001] one way trip" do
   home_page = expect_page HomePage
   home_page.login
   # . . .
   click_link("SIGN-OFF")
   goto_page("/")
end


test "[002] round trip" do
  home_page = expect_page HomePage
  home_page.login
  # . . .
  click_link("SIGN-OFF")
  goto_page("/")
end

Remove duplications

There are obvious duplications in test scripts. RSpec framework allows users to set operations before or after each test case execution.

Select the first two lines (login function) then press 'Shift + F7' to perform 'Move code' refactoring.

Figure 6. Refactoring 'Move code'

Select '2 Move to before(:each)' to move the operations into

before(:each) do
  home_page = expect_page HomePage
  home_page.login
end

As the name suggests, these two operations will be executed before each test case, so that the first two statements in Test Case 002 are not needed any more. And we can perform similar refactoring to create 'after(:each)' section.

after(:each) do
click_link("SIGN-OFF")
goto_page("/")
end

Final version

Here are a complete (refactored) test scripts for Test Case 001 and Test Case 002.

load File.dirname(__FILE__) + '/test_helper.rb'


  test_suite "Complete Test Script" do
    include TestHelper


    before(:all) do
      open_browser "http://newtours.demoaut.com"
    end


    before(:each) do
      home_page = expect_page HomePage
      home_page.login
    end


    after(:each) do
      click_link("SIGN-OFF")
      goto_page("/")
    end


    test "[001] one way trip" do
      select_flight_page = expect_page SelectFlightPage
      select_flight_page.select_trip_oneway
      select_flight_page.select_from_new_york
      select_flight_page.select_to_sydney
      select_flight_page.click_continue
      assert_text_present("New York to Sydney")
    end


    test "[002] round trip" do
      select_flight_page = expect_page SelectFlightPage
      select_flight_page.select_trip_round
      select_flight_page.select_from_new_york
      select_flight_page.select_to_sydney
      select_flight_page.click_continue
      assert_text_present("New York to Sydney")
      assert_text_present("Sydney to New York")
    end


  end

Coping with changes

We are not living in a perfect world. Things do change frequently in software development world. Fortunately, the above work makes test scripts not only more readable, but also easier to cope with changes.

1. Customers change terminologies

As we know, it is a good practice to speak same language in a project, even in test scripts. For instance, customers now prefer using term "Return Trip" rather than "Round Trip". With refactored test scripts, it can be done in seconds.

Move caret to the function 'select_trip_round' in 'SelectFlightPage' (pages\select_flight_page.rb), Select 'Rename ...' under 'Refactoring' menu (Shift+F6)

Figure 7. 'Refactor' menu - 'Rename'

Then enter new function name: 'select_return_trip'.

Figure 8. 'Rename Function' dialog

The references of 'select_trip_round' in test script file are updated with

select_flight_page.select_return_trip

2. Application Changes

Application changes (by programmers) are more common. For instance, a programmer changed the flight selection page for some reason, the attribute to identify the departure city has changed (in HTML) from

<select name="fromPort">
to
<select name="departurePort">

Although no visible changes from users' point of view, the test scripts (any test cases using that page) are now broken. It can be a quite tedious and error prone job if you are using recorded script directly as your test scripts.

Navigate to 'select_from_new_york' in 'SelectFlightPage' (Ctrl+T select 'select_flight_page', Ctrl+F12 then select 'select_from_xx'), and change 'fromPort' to 'departurePort'.

def select_from_new_york
  select_option("departurePort", "New York") # from 'fromPort'
end

That's not too hard!

Summary

In this article, we introduced using page objects in automated functional testing to make test scripts easy to understand and maintain. Through a real example, we demonstrated various refactorings using iTest2 IDE to improve the test scripts.

References

Fowler, Martin, et al. Refactoring: Improving the design of existing code, Reading, Mass.: Addison-Wesley, 1999