Introduction
The Tellurium Automated Testing Framework (Tellurium) is a framework for testing web applications, which was started in June 2007 by Jian Fang and became an open source project on Google Code in June 2008. It is released on a regular basis and is currently at 0.7.0.The core of the project was started over two years ago and quickly spawned multiple sub-projects including: UDL, Core, Engine, Widget extensions, Maven Archetypes, Trump, Tellurium IDE, TelluriumWorks, and reference projects.
The framework was developed from the Selenium framework, but with a different testing concept. Most existing web testing frameworks, like Selenium, primarily focus on individual UI elements. Tellurium on the other hand, treats the whole UI element as a widget; calling the element a UI module.
Taking the Google search UI as an example, it is represented in Tellurium as follows:
ui.Container(uid: "GoogleSearchModule", clocator: [tag: "td"]){
InputBox(uid: "Input", clocator: [title: "Google Search"])
SubmitButton(uid: "Search", clocator: [name: "btnG", value: "Google
Search"])
SubmitButton(uid: "ImFeelingLucky", clocator: [value: "I'm Feeling
Lucky"])
}
As shown in this example, the UI module is a set of nested UI elements with tags and attributes. The adoption of the UI module makes Tellurium expressive, and robust to changes. It is also easy to represent dynamic web content, and easy to maintain.
The framework comprises the following components:
- Trump - A Firefox plugin, properly the Tellurium UI Module Plugin , that automatically generates the UI module after a user selects the UI elements from the web page being tested.
- Tellurium IDE – A Firefox plugin that records user actions and generates Tellurium test scripts, including UI module definitions, actions, and assertions. The scripts are written in Groovy.
- TelluriumWorks – A standalone Java Swing application used to edit and run Tellurium test scripts. An IDE plugin for IntelliJ IDEA is in development.
- JavaScript Widget Extensions - Extensions for popular JavaScript frameworks such as Dojo and jQuery UI. This allows users to include the published Tellurium jar file and then treat the UI widget as a regular Tellurium object in the UI module definition.
Features
The main features are:
- The UI module clearly represents the UI being tested. In Tellurium's test code, locators are not used directly. The object uids are used to reference UI elements, which are expressive.
For example:
type "GoogleSearchModule.Input", "Tellurium test"
click "GoogleSearchModule.Search" - UI attributes are used to describe the UI instead of fixed locators. The actual locators are generated at runtime. If the attributes are changed, new runtime locators are generated by the framework. Tellurium then self-adapts to UI changes as necessary.
The Santa algorithm in Tellurium 0.7.0 further improves the test robustness by locating the whole UI module in a single attempt. A UI module partial match mechanism is then used to adapt to attribute changes up to a certain level. - The Tellurium UI templates and the Tellurium UID Description Language (UDL) are used to represent dynamic web content.
- The framework enforces the separation of UI modules from the test code, allowing easy refactoring.
For example, the UI and the corresponding methods are defined in a separate Groovy class. In this way, the test code is decoupled from the UI module.
In addition the framework:
- Uses abstract UI objects to encapsulate web UI elements
- Supports widgets for re-usability
- Offers a DSL for UI definition, actions, and testing
- Supports Group locating to locate a collection of UI components in one attempt
- Includes CSS selector support to improve test speed in IE
- Has Locator caching and command bundles to improve test speed
- Supports Data-driven test support
Comparing Selenium and Tellurium
The Selenium web testing framework is one of the most popular open source automated web testing frameworks. It is a ground-breaking framework offering many unique features and advantages such as: browser-based testing, Selenium Grid, and "record and replay" of user interactions with the Selenium IDE.
However, Selenium has some issues. Take the following test code for example:
setUp("http://www.google.com/", "*chrome");
selenium.open("/");
selenium.type("q", "Selenium test");
selenium.click("//input[@value='Google Search' and @type='button']");
If one were not familiar with the Google search page, could one tell what the UI of the page looked like based on that code? What does the locator q mean in this instance?
What if the XPath //input[@value='Google Search' and @type='button']became invalid due to changes on the web? More than likely, the test code would have to be reviewed in its entirely to locate the lines that needed to be updated.
What if there are tens or hundreds of locators in the test code? Creating the test code using the Selenium IDE may be easy to use initially, but it is difficult to generalize and refactor.
Refactoring is a more tedious procedure than generating new test code from scratch. The reason for this is that hard-coded locators tightly coupled with the test code are being used. Maintaining the code can be difficult because the test code is not structured.
Selenium is a good framework when it acts as a low level web test driving framework. However, it requires a lot of effort to create robust test code.
Tellurium was built with the goal of solving most of these issues with Selenium.
Tellurium is also designed to address other weak points of Selenium. For example, IE performance is a prominent issue for Selenium. The Tellurium solution to this situation is to use CSS selectors as the default locators. The locators are automatically generated from the UI module, improving the test speed.
Test speed is further improved with the Tellurium UI module caching and command bundling using the new Tellurium Engine. Tellurium also assists in the testing of Ajax applications: The List and Table Tellurium UI objects are used to represent dynamic web content at runtime. The option object is used to represent two different UIs at runtime for the same web element.
Like Selenium, Tellurium is used to test any web applications based on the HTML DOM structure.
Testing Approach
Tellurium takes a new approach to automated web-testing through the concept of the UI module. Objects are used to encapsulate web UI elements so that manually generalizing and refactoring of the UI locators is not required. The UI module is simply a composite UI object consisting of nested basic UI objects.
The framework runs in two modes. The first mode is to work as a wrapper to the Selenium framework. That is to say, the Tellurium core generates the runtime locator based on the UI object's attributes in a UI module. The generated runtime locator is then passed in the Selenium call to the Selenium core with Tellurium extensions.
Tellurium is also developing its own test driving engine, the Tellurium Engine, to better and more efficiently support UI modules.
- First, the Tellurium Core converts the UI module into a JSON representation.
- The JSON representation is then passed to the Tellurium Engine for the first time when the UI module is used.
- The Tellurium Engine then uses the Santa algorithm to locate the whole UI module and put it into a cache.
- For the subsequent calls, the cached UI module is used instead of re-locating them again.
- In addition, the Tellurium core combines multiple commands into one batch called a macro command and then sends the batch to the Tellurium Engine in one call. This reduces round trip latency.
The following example, which uses the issue search UI on the project web site, illustrates the idea.
We start by defining the UI module for the issue search UI
ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) { Selector(uid: "issueType", clocator: [name: "can", id: "can", direct: "true"]) TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"]) InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"]) SubmitButton(uid: "searchButton", clocator: [value: "Search", direct: "true"]) }
The following test method is used:
public void searchIssue(String type, String issue){ select "issueSearch.issueType", type keyType "issueSearch.searchBox", issue click "issueSearch.searchButton" waitForPageToLoad 30000 }
If for some reason, the Selector is changed to an input box, then we just update the UI module accordingly
ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) { InputBox(uid: "issueType", clocator: [name: "can", direct: "true"]) TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"]) InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"]) SubmitButton(uid: "searchButton", clocator: [value: "Search", direct: "true"]) }
then change the command:
select "issueSearch.issueType", type
to:
type "issueSearch.issueType", type
and the rest remains the same.
When there is dynamic web content, taking the Google Books website as an example, the UI includes a list of book categories with a list of books inside each category. The UI module for this UI is surprisingly simple to use as follows:
ui.Container(uid: "GoogleBooksList", clocator: [tag: "table", id: "hp_table"]) {
List(uid: "subcategory", clocator: [tag: "td", class: "sidebar"], separator:
"div") {
Container(uid: "{all}") {
TextBox(uid: "title", clocator: [tag: "div", class: "sub_cat_title"])
List(uid: "links", separator: "p") {
UrlLink(uid: "{all}", clocator: [:])
}
}
}}
The Tellurium UID description language (UDL) provides more flexibility to define dynamic web content. Let us see a complex example.
ui.StandardTable(uid: "GT", clocator: [id: "xyz"], ht: "tbody"){
TextBox(uid: "{header: first} as One", clocator: [tag: "th", text: "one"], self:
true)
TextBox(uid: "{header: 2} as Two", clocator: [tag: "th", text: "two"], self: true)
TextBox(uid: "{header: last} as Three", clocator: [tag: "th", text: "three"],
self: true)
TextBox(uid: "{row: 1, column -> One} as A", clocator: [tag: "div", class: "abc"])
Container(uid: "{row: 1, column -> Two} as B"){
InputBox(uid: "Input", clocator: [tag: "input", class: "123"])
Container(uid: "Some", clocator: [tag: "div", class: "someclass"]){
Span(uid: "Span", clocator: [tag: "span", class: "x"])
UrlLink(uid: "Link", clocator: [:])
}
}
TextBox(uid: "{row: 1, column -> Three} as Hello", clocator: [tag: "td"], self:
true)
}
In the above example, we use meta data "first", number, and "last" to indicate the header positions. The meta data "{row: 1, column -> One} as A" means the UI element, a TextBox in our case, is in row 1 and the same column as where the header "One" is. The test code is very clean, for example:
getText "GT.A"
keyType "GT.B.Input", input
click "GT.B.Some.Link"
waitForPageToLoad 30000
Future Plans
Tellurium is a young and innovative framework with many novel ideas from both the development team and the user community. There are many areas Tellurium would like to develop:
- Tellurium 0.7.0 has implemented a new test driving engine using jQuery. The main features of the Engine are: UI module group locating, UI module caching, Command bundle processing, Selenium APIs re-implemented in jQuery, and new Tellurium APIs. Tellurium will continue to develop the new Engine to its maturity.
- Tellurium UI module plugin 0.8.0 RC1 is just released and it includes many new features. Tellurium IDE release candidate is also out to record and generate test scripts. They are key to the success of Tellurium and they will continue to be improved upon. Besides Trump and Tellurium IDE, Tellurium is planning to improve TelluriumWorks so that it will edit, complete syntax checks, and run Tellurium DSL test scripts.
- Tellurium as a cloud testing tool is another very important future development. The project team is planning to rethink the architecture to make it more straightforward to execute tests in parallel. It is very challenging to exploit peer-to-peer techniques to make the test servers capable of self-organizing and self-coordinating in the cloud environment with the least management effort.
Other areas of the framework to be developed include:
- The creation of reusable Dojo, ExtJS, and jQuery UI Tellurium widgets. This will allow other people to reuse the widgets simply by including the jar files in their projects.
- Behavior Driven Testing support.
- Testing flow support.
- Web security testing.
- Support for other languages such as Ruby.
About The Author
Jian Fang graduated from Georgia Institute of Technology with a Ph.D. degree in Electrical and Computer Engineering. He works as a senior software engineer in a company in the IT industry and mainly focuses on the design and implementation of enterprise applications. He is the creator of the Tellurium Automated Testing Framework.