BT

Introducing the Tellurium Automated Testing Framework

Posted by Jian Fang on Sep 21, 2010 |

Introduction

The Tellurium Automated Testing Framework (Tellurium) is a framework for testing web applications, which was started in June 2007 by Jian Fang and became an open source project on Google Code in June 2008. It is released on a regular basis and is currently at 0.7.0.
The core of the project was started over two years ago and quickly spawned multiple sub-projects including: UDL, Core, Engine, Widget extensions, Maven Archetypes, Trump, Tellurium IDE, TelluriumWorks, and reference projects.
The framework was developed from the Selenium framework, but with a different testing concept. Most existing web testing frameworks, like Selenium, primarily focus on individual UI elements. Tellurium on the other hand, treats the whole UI element as a widget; calling the element a UI module.

Taking the Google search UI as an example, it is represented in Tellurium as follows:
ui.Container(uid: "GoogleSearchModule", clocator: [tag: "td"]){  
InputBox(uid: "Input", clocator: [title: "Google Search"])  
SubmitButton(uid: "Search", clocator: [name: "btnG", value: "Google
Search"])  
SubmitButton(uid: "ImFeelingLucky", clocator: [value: "I'm Feeling
Lucky"])
}

As shown in this example, the UI module is a set of nested UI elements with tags and attributes. The adoption of the UI module makes Tellurium expressive, and robust to changes. It is also easy to represent dynamic web content, and easy to maintain.

The framework comprises the following components:

  1. Trump - A Firefox plugin, properly the Tellurium UI Module Plugin , that automatically generates the UI module after a user selects the UI elements from the web page being tested.
  2. Tellurium IDEA Firefox plugin that records user actions and generates Tellurium test scripts, including UI module definitions, actions, and assertions. The scripts are written in Groovy.
  3. TelluriumWorks – A standalone Java Swing application used to edit and run Tellurium test scripts. An IDE plugin for IntelliJ IDEA is in development.
  4. JavaScript Widget Extensions - Extensions for popular JavaScript frameworks such as Dojo and jQuery UI.  This allows users to include the published Tellurium jar file and then treat the UI widget as a regular Tellurium object in the UI module definition.

Features

The main features are:

  1. The UI module clearly represents the UI being tested. In Tellurium's test code, locators are not used directly. The object uids are used to reference UI elements, which are expressive.

    For example:

    type "GoogleSearchModule.Input", "Tellurium test"
    click "GoogleSearchModule.Search"
  2. UI attributes are used to describe the UI instead of fixed locators. The actual locators are generated at runtime. If the attributes are changed, new runtime locators are generated by the framework. Tellurium then self-adapts to UI changes as necessary.
    The Santa algorithm in Tellurium 0.7.0 further improves the test robustness by locating the whole UI module in a single attempt. A UI module partial match mechanism is then used to adapt to attribute changes up to a certain level.
  3. The Tellurium UI templates and the Tellurium UID Description Language (UDL) are used to represent dynamic web content.
  4. The framework enforces the separation of UI modules from the test code, allowing easy refactoring.
    For example, the UI and the corresponding methods are defined in a separate Groovy class.  In this way, the test code is decoupled from the UI module.

In addition the framework:

  • Uses abstract UI objects to encapsulate web UI elements
  • Supports widgets for re-usability
  • Offers a DSL for UI definition, actions, and testing
  • Supports Group locating to locate a collection of UI components in one attempt
  • Includes CSS selector support to improve test speed in IE
  • Has Locator caching and command bundles to improve test speed
  • Supports Data-driven test support

Comparing Selenium and Tellurium

The Selenium web testing framework is one of the most popular open source automated web testing frameworks. It is a ground-breaking framework offering many unique features and advantages such as: browser-based testing, Selenium Grid, and "record and replay" of user interactions with the Selenium IDE.

However, Selenium has some issues. Take the following test code for example:

setUp("http://www.google.com/", "*chrome"); 
selenium.open("/");
selenium.type("q", "Selenium test");
selenium.click("//input[@value='Google Search' and @type='button']");

If one were not familiar with the Google search page, could one tell what the UI of the page looked like based on that code?   What does the locator q mean in this instance? 

What if the XPath //input[@value='Google Search' and @type='button']became invalid due to changes on the web?  More than likely, the test code would have to be reviewed in its entirely to locate the lines that needed to be updated.

What if there are tens or hundreds of locators in the test code?  Creating the test code using the Selenium IDE may be easy to use initially, but it is difficult to generalize and refactor.  

Refactoring is a more tedious procedure than generating new test code from scratch.  The reason for this is that hard-coded locators tightly coupled with the test code are being used.  Maintaining the code can be difficult because the test code is not structured.

Selenium is a good framework when it acts as a low level web test driving framework. However, it requires a lot of effort to create robust test code.  

Tellurium was built with the goal of solving most of these issues with Selenium. 

Tellurium is also designed to address other weak points of Selenium. For example, IE performance is a prominent issue for Selenium. The Tellurium solution to this situation is to use CSS selectors as the default locators. The locators are automatically generated from the UI module, improving the test speed.

Test speed is further improved with the Tellurium UI module caching and command bundling using the new Tellurium Engine. Tellurium also assists in the testing of Ajax applications: The List and Table Tellurium UI objects are used to represent dynamic web content at runtime. The option object is used to represent two different UIs at runtime for the same web element.

Like Selenium, Tellurium is used to test any web applications based on the HTML DOM structure.

Testing Approach

Tellurium takes a new approach to automated web-testing through the concept of the UI module.  Objects are used to encapsulate web UI elements so that manually generalizing and refactoring of the UI locators is not required.  The UI module is simply a composite UI object consisting of nested basic UI objects. 

The framework runs in two modes. The first mode is to work as a wrapper to the Selenium framework. That is to say, the Tellurium core generates the runtime locator based on the UI object's attributes in a UI module.  The generated runtime locator is then passed in the Selenium call to the Selenium core with Tellurium extensions.

Tellurium is also developing its own test driving engine, the Tellurium Engine, to better and more efficiently support UI modules.

  1. First, the Tellurium Core converts the UI module into a JSON representation.
  2. The JSON representation is then passed to the Tellurium Engine for the first time when the UI module is used.
  3. The Tellurium Engine then uses the Santa algorithm to locate the whole UI module and put it into a cache.  
  4. For the subsequent calls, the cached UI module is used instead of re-locating them again.
  5. In addition, the Tellurium core combines multiple commands into one batch called a macro command and then sends the batch to the Tellurium Engine in one call.  This reduces round trip latency.

The following example, which uses the issue search UI on the project web site, illustrates the idea.

We start by defining the UI module for the issue search UI

ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) {
	Selector(uid: "issueType", clocator: [name: "can", id: "can", direct:
	"true"])
	TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"])
	InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"])
	SubmitButton(uid: "searchButton", clocator: [value: "Search", direct:
	"true"])
}

The following test method is used:

public void searchIssue(String type, String issue){
	select "issueSearch.issueType", type
	keyType "issueSearch.searchBox", issue
	click "issueSearch.searchButton"
	waitForPageToLoad 30000
}

If for some reason, the Selector is changed to an input box, then we just update the UI module accordingly

ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) {
	InputBox(uid: "issueType", clocator: [name: "can", direct: "true"])
	TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"])
	InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"])
	SubmitButton(uid: "searchButton", clocator: [value: "Search", direct: "true"])
}

then change the command:

select "issueSearch.issueType", type

to:

type "issueSearch.issueType", type

and the rest remains the same.

When there is dynamic web content, taking the Google Books website as an example, the UI includes a list of book categories with a list of books inside each category. The UI module for this UI is surprisingly simple to use as follows:

ui.Container(uid: "GoogleBooksList", clocator: [tag: "table", id: "hp_table"]) {
List(uid: "subcategory", clocator: [tag: "td", class: "sidebar"], separator:
"div") {
Container(uid: "{all}") {
TextBox(uid: "title", clocator: [tag: "div", class: "sub_cat_title"])
List(uid: "links", separator: "p") {
UrlLink(uid: "{all}", clocator: [:])
}
}
}}

The Tellurium UID description language (UDL) provides more flexibility to define dynamic web content. Let us see a complex example.

ui.StandardTable(uid: "GT", clocator: [id: "xyz"], ht: "tbody"){
TextBox(uid: "{header: first} as One", clocator: [tag: "th", text: "one"], self:
true)
TextBox(uid: "{header: 2} as Two", clocator: [tag: "th", text: "two"], self: true)
TextBox(uid: "{header: last} as Three", clocator: [tag: "th", text: "three"],
self: true)
TextBox(uid: "{row: 1, column -> One} as A", clocator: [tag: "div", class: "abc"])
Container(uid: "{row: 1, column -> Two} as B"){
InputBox(uid: "Input", clocator: [tag: "input", class: "123"])
Container(uid: "Some", clocator: [tag: "div", class: "someclass"]){
Span(uid: "Span", clocator: [tag: "span", class: "x"])
UrlLink(uid: "Link", clocator: [:])
}
}
TextBox(uid: "{row: 1, column -> Three} as Hello", clocator: [tag: "td"], self:
true)
}

In the above example, we use meta data "first", number, and "last" to indicate the header positions. The meta data "{row: 1, column -> One} as A" means the UI element, a TextBox in our case, is in row 1 and the same column as where the header "One" is. The test code is very clean, for example:

getText "GT.A"
keyType "GT.B.Input", input
click "GT.B.Some.Link"
waitForPageToLoad 30000

Future Plans

Tellurium is a young and innovative framework with many novel ideas from both the development team and the user community.  There are many areas Tellurium would like to develop:

  1. Tellurium 0.7.0 has implemented a new test driving engine using jQuery.  The main features of the Engine are: UI module group locating, UI module caching, Command bundle processing, Selenium APIs re-implemented in jQuery, and new Tellurium APIs.  Tellurium will continue to develop the new Engine to its maturity.
  2. Tellurium UI module plugin 0.8.0 RC1 is just released and it includes many new features. Tellurium IDE release candidate is also out to record and generate test scripts. They are key to the success of Tellurium and they will continue to be improved upon. Besides Trump and Tellurium IDE, Tellurium is planning to improve TelluriumWorks so that it will edit, complete syntax checks, and run Tellurium DSL test scripts.
  3. Tellurium as a cloud testing tool is another very important future development. The project team is planning to rethink the architecture to make it more straightforward to execute tests in parallel.  It is very challenging to exploit peer-to-peer techniques to make the test servers capable of self-organizing and self-coordinating in the cloud environment with the least management effort.

Other areas of the framework to be developed include:

  • The creation of reusable Dojo, ExtJS, and jQuery UI Tellurium widgets. This will allow other people to reuse the widgets simply by including the jar files in their projects.
  • Behavior Driven Testing support.
  • Testing flow support. 
  • Web security testing.
  • Support for other languages such as Ruby.

About The Author

Jian Fang graduated from Georgia Institute of Technology with a Ph.D. degree in Electrical and Computer Engineering. He works as a senior software engineer in a company in the IT industry and mainly focuses on the design and implementation of enterprise applications. He is the creator of the Tellurium Automated Testing Framework.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

tellurium rocks!!!!! by Haroon Rasheed

We have been using Tellurium for some time now, our tests are more expressive now with low cost of maintenance.

Some Resources for Tellurium by John Fang

Tellurium by Jade Lindquist

We're using tellurium's tests and data-driven tests for our testing. With Tellurium, we've been able to overcome issues that we had with Selenium tests on web pages with Ajax. The Tellurium developers are the most responsive open source developers that I've interacted with. Awesome software!

Jade

Verbose? by Vikas Hazrati

May be I am jumping the gun here, since I have not given it a try yet or may be I am too used to Selenium but the code examples for Tellurium look pretty verbose syntactically as compared to Selenium

Re: Verbose? by John Fang

Consider the following command in Selenium:

selenium.click("//input[@value='Google Search' and @type='button']");

In Tellurium, the command is

click "Google.Search"

Which one is verbose?

Responsive community by Jonathan Share

Not only is Tellurium a great framework for doing web-testing but the community, although small, is very responsive to the needs of the users. I've seen a very quick response to problems I've reported via the mailing list.

Re: Verbose? by Behrang Saeedzadeh

Compare full working snippets. Your example is misleading. You haven't shown the code necessary to create the "Google" module in Tellurium.

Re: Tellurium by Behrang Saeedzadeh

we've been able to overcome issues that we had with Selenium tests on web pages with Ajax


Could you please elaborate? In particular, what type of Ajax testing issues are you referring to?

Re: Verbose? by John Fang

I don't think InfoQ wants me to post all code, but most stuff has already been posted. Here is the full UI module class:

public class GoogleSearchModule extends DslContext {

public void defineUi() {
ui.Container(uid: "GoogleSearchModule", clocator: [tag: "td"]){
InputBox(uid: "Input", clocator: [title: "Google Search"])
SubmitButton(uid: "Search", clocator: [name: "btnG", value: "Google Search"])
SubmitButton(uid: "ImFeelingLucky", clocator: [value: "I'm Feeling Lucky"])
}
}

public void doGoogleSearch(String input) {
type "GoogleSearchModule.Input", input
click "GoogleSearchModule.Search"
waitForPageToLoad 30000
}

}



Compare full working snippets. Your example is misleading. You haven't shown the code necessary to create the "Google" module in Tellurium.

Re: Verbose? by Behrang Saeedzadeh

And isn't that verbose?

Re: Verbose? by John Fang

You mean the UI module definition? Isn't it very expressive about
the UIs under testing? Once you define it, you always use UIDs to
reference UI elements, which is not verbose.

And isn't that verbose?

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

11 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT