Refactoring Automated Functional Test Scripts with iTest2
Automated test scripts are known difficult to maintain. With wide adoption of agile methodologies in enterprise software projects, one of its core practices: automated functional testing proves its value, at the same time provides challenges to projects. Traditional record-n-playback testing tools may help creating a set of test scripts quickly, but often end up unmaintainable. The reason: application changes.
In programming world, 'refactoring' (a process improves software internal structure without changing its behaviour) has become a highly frequent used word among programmers. In simple words, programmers make code more readable and design more flexible during refactoring process. Experienced agile project managers allocate certain time for programmers to perform code refactoring or make it as part of process finishing user stories. Most of Integrated Development Environments (IDEs) come with support for various refactorings.
Testers who develop or maintain automated test scripts usually do not have that kind of luxury, but share the same needs to make automated test scripts readable and maintainable. It is difficult (the more test scripts, the more difficult it becomes) to get the test scripts back on track for releases with new features, bug fixes and software changes.
The objective and procedures of functional test refactoring is the same with code refactoring, but it has special characteristics:
- Target Audience
The end users of testing tool are testers, business analysts or even customers. The fact is that testers, business analysts and customers generally do not possess programming skills, and this changes the whole paradigm.
- Script Syntax
Code refactoring is mostly supported on compiled languages such as Java and C#. Functional test scripts, however, may be in a form of XML, proprietary vendor scripts, compiled languages or script languages (such as Ruby). Depending on the test framework, the use of refactoring varies.
- Refactorings specific to functional testing
While some common code refactorings, such as 'Rename', apply to functional test scripts, they are ones are specific for testing purposes, such as 'Move the scripts to run each test case".
A new functional testing tool: iTest2 IDE is designed for testers to develop and maintain automated test scripts with ease. iTest2 is written from ground up dedicated for web test automation, the test framework it supports is rWebUnit (an open source extension of popular Watir - Web App Testing in Ruby) in RSpec syntax.
The philosophy of iTest2 is: easy and simple. Trials showed testers without programming experiences could write their first automated test scripts averagely less than 10 minutes under mentoring. With iTest2, testers can develop, maintain and verify test scripts against functional requirements; developers can verify the feature is working on; business analysts/customers view test execution (in a real browser: IE or Firefox) to verify function requirements.
The test scripts created by iTest2 can be executed from command line and integrated with continuous build servers.
An example worth thousands of words. We will walk through complete steps from creating two cases to making them readable and maintainable by using refactoring tools in iTest2.
For our exercise, we develop typical yet simple web test scripts for Mercury's NewTour web site.
|Test Data:||User Login: agileway / agileway|
|Test Case 001:||A registered customer can select a one way flight from New York to Sydney|
|Test Case 002:||A registered customer can select a round trip flight from New York to Sydney|
|Test Script Framework:||rWebUnit (an extension of Watir, open-source)|
|Test Execution:||Command line or iTest2 IDE|
|Test Editor/Tools:||iTest2 IDE|
Create test case 001
1. Create a project
Firstly, we create an iTest2 project, specify the site URL, and a sample test script file will be created as below:
load File.dirname(__FILE__) + '/test_helper.rb' test_suite "TODO" do include TestHelper before(:all) do open_browser "http://newtours.demoaut.com" end test "your test case name" do # add your test scripts here end end
2. Use iTest2Recorder to record test scripts for Test Case 001
We use iTest2 Recorder, a Firefox add-on records user operations in Firefox browser into executable test scripts.
enter_text("userName", "agileway") enter_text("password", "agileway") click_button_with_image("btn_signin.gif") click_radio_option("tripType", "oneway") select_option("fromPort", "New York") select_option("toPort", "Sydney") click_button_with_image("continue.gif") assert_text_present("New York to Sydney")
3. Paste recorded test script in a test script file, and run it.
# ... test " one way trip" do enter_text("userName", "agileway") enter_text("password", "agileway") click_button_with_image("btn_signin.gif") click_radio_option("tripType", "oneway") select_option("fromPort", "New York") select_option("toPort", "Sydney") click_button_with_image("continue.gif") assert_text_present("New York to Sydney") end
Now run the test case (right click and select 'Run  one way trip'), it passed!
Refactor to use page objects
The above test scripts work and rWebUnit syntax is quite readable. Some might question the needs for refactoring, and what is 'using pages'?
First of all, test scripts in current format are not easy to maintain. Let's say we now have hundreds of automated test scripts, new released software changed user authentication to use customer's email as username to login, which in turn means we need to change to use 'email' instead of 'userName' in test scripts. Performing search and replace on hundreds of files does not sound like a good solution. Also project members like to speak common vocabulary within the project, which has a fancy name: Domain Specific Language (DSL). It is nice to see them used in test scripts as well.
It can be done using page objects. A page in our context represents a web page logically, it contains operations provided to end user on that page. For example, the home page in our example has three operations: 'enter user name', 'enter password' and 'click login button'. 'Refactor to use pages' is a process to extract operations into specific page objects, and it is made quite easy to do so with refactoring support in iTest2.
1. Extract to HomePage
The login function is on the home page, and we will make it so. As user login is a well understood function, we make 3 lines of statements (enter username, password and clicking login button) into one operation. Select those 3 lines, then click 'Extract Page ...' under 'Refactoring' menu (keyboard shortcut: Ctrl+Alt+G).
Figure 1. 'Refactor' menu - 'Extract Page...'
This opens a window like below to for you to enter page name and function name, we enter 'HomePage' and 'login' respectively.
Figure 2. 'Extract Page' dialog box
The selected statements (3 lines) are now replaced with
home_page = expect_page HomePage home_page.login
A new file 'pages\home_page.rb' is created with the following content: class HomePage < RWebUnit::AbstractWebPage
class HomePage < RWebUnit::AbstractWebPage def initialize(browser) super(browser, "") # TODO: add identity text (in quotes) end def login enter_text("userName", "agileway") enter_text("password", "agileway") click_button_with_image("btn_signin.gif") end end
Run the test case again, it shall still pass.
Note: As Martin Fowler pointed out: the rhythm of refactoring: test, small change, test, small change, test, small change. It is that rhythm that allows refactoring to move quickly and safely.
2. Extract to SelectFlightPage
After login successfully, customers land at flight selection page. Different from login page, every operation here is more likely to be updated independently by developers, so we extract each operation to a new function. Move caret to the line
Perform another 'Extract to Page..." refactoring (Ctrl+Alt+G), enter "SelectFlightPage" and "select_trip_oneway" for new page and function name.
select_flight_page = expect_page SelectFlightPage select_flight_page.select_trip_oneway
3. Continue extract more operations into SelectFlightPage
Continue performing refactorings for the remaining operations to 'SelectFlightPage'': 'select_from_new_york', 'select_to_sydney', and 'click_continue'.
test " one way trip" do home_page = expect_page HomePage home_page.login select_flight_page = expect_page SelectFlightPage select_flight_page.select_trip_oneway select_flight_page.select_from_new_york select_flight_page.select_to_sydney select_flight_page.click_continue assert_text_present("New York to Sydney") end
As always, we run the test case again.
Write test case 002
Since we now have two pages ('HomePage' and 'SelectFlightPage') from refactoring test case 001, writing test case 002 will be a lot easier (by reusing them).
1. Using existing HomePage
iTest2 IDE has built-in support for page objects, typing "ep" and pressing 'Tab' key (called 'snippets') will expand to 'expect_page' and populate all known pages for selection.
Figure 3. Auto-complete pages
To use HomePage, we need to get a handle to it (in programming world, it is called 'variable'). Perform "Introduce Page Variable" refactoring (Ctrl+Alt+V) to create the variable.
Figure 4. 'Refactor' menu - 'Introduce Page Variable'
home_page = expect_page HomePage
Now type "homepage." in next statement, the functions defined in the page class will show up for you to choose.
Figure 5. Page function lookup
2. Add dedicated operation for Test Case 2
Test Case 002 is quite similar to Test Case 001, the differences are trip type selection and assertions. With the help of the recorder, we can identify the new operation:
Then refactor it into a new function in SelectFlightPage
Here it is
test " round trip" do home_page = expect_page HomePage home_page.login select_flight_page = expect_page SelectFlightPage select_flight_page.select_trip_round select_flight_page.select_from_new_york select_flight_page.select_to_sydney select_flight_page.click_continue assert_text_present("New York to Sydney") assert_text_present("Sydney to New York") end
Run the test scripts for Test Case 2 (Right click any line in test case 2, and select 'Run ...'), it passed!
Reset application to initial state
But wait, we are not quite finished yet. Test Case 1 passed, Test Case 2 passed, running them together got an error on Test Case 2, why?
We did not reset the web application back to initial state, the user remains signed in after finishing execution of Test Case 001. To make tests independent from each other, we make sure the test execution starting with sign-in and end with sign-off.
test " one way trip" do home_page = expect_page HomePage home_page.login # . . . click_link("SIGN-OFF") goto_page("/") end test " round trip" do home_page = expect_page HomePage home_page.login # . . . click_link("SIGN-OFF") goto_page("/") end
There are obvious duplications in test scripts. RSpec framework allows users to set operations before or after each test case execution.
Select the first two lines (login function) then press 'Shift + F7' to perform 'Move code' refactoring.
Figure 6. Refactoring 'Move code'
Select '2 Move to before(:each)' to move the operations into
before(:each) do home_page = expect_page HomePage home_page.login end
As the name suggests, these two operations will be executed before each test case, so that the first two statements in Test Case 002 are not needed any more. And we can perform similar refactoring to create 'after(:each)' section.
Here are a complete (refactored) test scripts for Test Case 001 and Test Case 002.
load File.dirname(__FILE__) + '/test_helper.rb' test_suite "Complete Test Script" do include TestHelper before(:all) do open_browser "http://newtours.demoaut.com" end before(:each) do home_page = expect_page HomePage home_page.login end after(:each) do click_link("SIGN-OFF") goto_page("/") end test " one way trip" do select_flight_page = expect_page SelectFlightPage select_flight_page.select_trip_oneway select_flight_page.select_from_new_york select_flight_page.select_to_sydney select_flight_page.click_continue assert_text_present("New York to Sydney") end test " round trip" do select_flight_page = expect_page SelectFlightPage select_flight_page.select_trip_round select_flight_page.select_from_new_york select_flight_page.select_to_sydney select_flight_page.click_continue assert_text_present("New York to Sydney") assert_text_present("Sydney to New York") end end
Coping with changes
We are not living in a perfect world. Things do change frequently in software development world. Fortunately, the above work makes test scripts not only more readable, but also easier to cope with changes.
1. Customers change terminologies
As we know, it is a good practice to speak same language in a project, even in test scripts. For instance, customers now prefer using term "Return Trip" rather than "Round Trip". With refactored test scripts, it can be done in seconds.
Move caret to the function 'select_trip_round' in 'SelectFlightPage' (pages\select_flight_page.rb), Select 'Rename ...' under 'Refactoring' menu (Shift+F6)
Figure 7. 'Refactor' menu - 'Rename'
Then enter new function name: 'select_return_trip'.
Figure 8. 'Rename Function' dialog
The references of 'select_trip_round' in test script file are updated with
2. Application Changes
Application changes (by programmers) are more common. For instance, a programmer changed the flight selection page for some reason, the attribute to identify the departure city has changed (in HTML) from
<select name="fromPort"> to <select name="departurePort">
Although no visible changes from users' point of view, the test scripts (any test cases using that page) are now broken. It can be a quite tedious and error prone job if you are using recorded script directly as your test scripts.
Navigate to 'select_from_new_york' in 'SelectFlightPage' (Ctrl+T select 'select_flight_page', Ctrl+F12 then select 'select_from_xx'), and change 'fromPort' to 'departurePort'.
def select_from_new_york select_option("departurePort", "New York") # from 'fromPort' end
That's not too hard!
In this article, we introduced using page objects in automated functional testing to make test scripts easy to understand and maintain. Through a real example, we demonstrated various refactorings using iTest2 IDE to improve the test scripts.
Fowler, Martin, et al. Refactoring: Improving the design of existing code, Reading, Mass.: Addison-Wesley, 1999