Introducing the Tellurium Automated Testing Framework

Posted by Jian Fang on Sep 21, 2010

Sections

Architecture & Design,

Development

Topics

Java ,

Languages ,

Websphere ,

Programming ,

IBM ,

Application Servers ,

Agile in the Enterprise ,

Companies ,

Software Testing ,

TDD ,

Agile ,

Testing

Introduction

The Tellurium Automated Testing Framework (Tellurium) is a framework for testing web applications, which was started in June 2007 by Jian Fang and became an open source project on Google Code in June 2008. It is released on a regular basis and is currently at 0.7.0.
The core of the project was started over two years ago and quickly spawned multiple sub-projects including: UDL, Core, Engine, Widget extensions, Maven Archetypes, Trump, Tellurium IDE, TelluriumWorks, and reference projects.
The framework was developed from the Selenium framework, but with a different testing concept. Most existing web testing frameworks, like Selenium, primarily focus on individual UI elements. Tellurium on the other hand, treats the whole UI element as a widget; calling the element a UI module.

Taking the Google search UI as an example, it is represented in Tellurium as follows:

ui.Container(uid: "GoogleSearchModule", clocator: [tag: "td"]){  
	InputBox(uid: "Input", clocator: [title: "Google Search"])  
	SubmitButton(uid: "Search", clocator: [name: "btnG", value: "Google
Search"])  
	SubmitButton(uid: "ImFeelingLucky", clocator: [value: "I'm Feeling
 Lucky"])
} 

As shown in this example, the UI module is a set of nested UI elements with tags and attributes. The adoption of the UI module makes Tellurium expressive, and robust to changes. It is also easy to represent dynamic web content, and easy to maintain.

The framework comprises the following components:

1. Trump - A Firefox plugin, properly the Tellurium UI Module Plugin , that automatically generates the UI module after a user selects the UI elements from the web page being tested.
2. Tellurium IDE – A Firefox plugin that records user actions and generates Tellurium test scripts, including UI module definitions, actions, and assertions. The scripts are written in Groovy.
3. TelluriumWorks – A standalone Java Swing application used to edit and run Tellurium test scripts. An IDE plugin for IntelliJ IDEA is in development.
4. JavaScript Widget Extensions - Extensions for popular JavaScript frameworks such as Dojo and jQuery UI.  This allows users to include the published Tellurium jar file and then treat the UI widget as a regular Tellurium object in the UI module definition.

Features

The main features are:

1. The UI module clearly represents the UI being tested. In Tellurium's test code, locators are not used directly. The object uids are used to reference UI elements, which are expressive.

   For example:

   type "GoogleSearchModule.Input", "Tellurium test"
   click "GoogleSearchModule.Search"

2. UI attributes are used to describe the UI instead of fixed locators. The actual locators are generated at runtime. If the attributes are changed, new runtime locators are generated by the framework. Tellurium then self-adapts to UI changes as necessary.
   The Santa algorithm in Tellurium 0.7.0 further improves the test robustness by locating the whole UI module in a single attempt. A UI module partial match mechanism is then used to adapt to attribute changes up to a certain level.
3. The Tellurium UI templates and the Tellurium UID Description Language (UDL) are used to represent dynamic web content.
4. The framework enforces the separation of UI modules from the test code, allowing easy refactoring.
   For example, the UI and the corresponding methods are defined in a separate Groovy class.  In this way, the test code is decoupled from the UI module.

In addition the framework:

• Uses abstract UI objects to encapsulate web UI elements
• Supports widgets for re-usability
• Offers a DSL for UI definition, actions, and testing
• Supports Group locating to locate a collection of UI components in one attempt
• Includes CSS selector support to improve test speed in IE
• Has Locator caching and command bundles to improve test speed
• Supports Data-driven test support

Comparing Selenium and Tellurium

The Selenium web testing framework is one of the most popular open source automated web testing frameworks. It is a ground-breaking framework offering many unique features and advantages such as: browser-based testing, Selenium Grid, and "record and replay" of user interactions with the Selenium IDE.

However, Selenium has some issues. Take the following test code for example:

setUp("http://www.google.com/", "*chrome"); 
selenium.open("/"); 
selenium.type("q", "Selenium test"); 
selenium.click("//input[@value='Google Search' and @type='button']");

If one were not familiar with the Google search page, could one tell what the UI of the page looked like based on that code?   What does the locator q mean in this instance? 

What if the XPath //input[@value='Google Search' and @type='button']became invalid due to changes on the web?  More than likely, the test code would have to be reviewed in its entirely to locate the lines that needed to be updated.

What if there are tens or hundreds of locators in the test code?  Creating the test code using the Selenium IDE may be easy to use initially, but it is difficult to generalize and refactor.  

Refactoring is a more tedious procedure than generating new test code from scratch.  The reason for this is that hard-coded locators tightly coupled with the test code are being used.  Maintaining the code can be difficult because the test code is not structured.

Selenium is a good framework when it acts as a low level web test driving framework. However, it requires a lot of effort to create robust test code.  

Tellurium was built with the goal of solving most of these issues with Selenium. 

Tellurium is also designed to address other weak points of Selenium. For example, IE performance is a prominent issue for Selenium. The Tellurium solution to this situation is to use CSS selectors as the default locators. The locators are automatically generated from the UI module, improving the test speed.

Test speed is further improved with the Tellurium UI module caching and command bundling using the new Tellurium Engine. Tellurium also assists in the testing of Ajax applications: The List and Table Tellurium UI objects are used to represent dynamic web content at runtime. The option object is used to represent two different UIs at runtime for the same web element.

Like Selenium, Tellurium is used to test any web applications based on the HTML DOM structure.

Testing Approach

Tellurium takes a new approach to automated web-testing through the concept of the UI module.  Objects are used to encapsulate web UI elements so that manually generalizing and refactoring of the UI locators is not required.  The UI module is simply a composite UI object consisting of nested basic UI objects. 

The framework runs in two modes. The first mode is to work as a wrapper to the Selenium framework. That is to say, the Tellurium core generates the runtime locator based on the UI object's attributes in a UI module.  The generated runtime locator is then passed in the Selenium call to the Selenium core with Tellurium extensions.

Tellurium is also developing its own test driving engine, the Tellurium Engine, to better and more efficiently support UI modules.

1. First, the Tellurium Core converts the UI module into a JSON representation.
2. The JSON representation is then passed to the Tellurium Engine for the first time when the UI module is used.
3. The Tellurium Engine then uses the Santa algorithm to locate the whole UI module and put it into a cache.  
4. For the subsequent calls, the cached UI module is used instead of re-locating them again.
5. In addition, the Tellurium core combines multiple commands into one batch called a macro command and then sends the batch to the Tellurium Engine in one call.  This reduces round trip latency.

The following example, which uses the issue search UI on the project web site, illustrates the idea.

We start by defining the UI module for the issue search UI

ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) {
	Selector(uid: "issueType", clocator: [name: "can", id: "can", direct:
	"true"])
	TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"])
	InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"])
	SubmitButton(uid: "searchButton", clocator: [value: "Search", direct:
	"true"])
}

The following test method is used:

public void searchIssue(String type, String issue){
	select "issueSearch.issueType", type
	keyType "issueSearch.searchBox", issue
	click "issueSearch.searchButton"
	waitForPageToLoad 30000
}

If for some reason, the Selector is changed to an input box, then we just update the UI module accordingly

ui.Form(uid: "issueSearch", clocator: [action: "list", method: "GET"]) {
	InputBox(uid: "issueType", clocator: [name: "can", direct: "true"])
	TextBox(uid: "searchLabel", clocator: [tag: "span", text: "for"])
	InputBox(uid: "searchBox", clocator: [type: "text", name: "q", id: "q"])
	SubmitButton(uid: "searchButton", clocator: [value: "Search", direct: "true"])
}

then change the command:

select "issueSearch.issueType", type

to:

type "issueSearch.issueType", type

and the rest remains the same.

When there is dynamic web content, taking the Google Books website as an example, the UI includes a list of book categories with a list of books inside each category. The UI module for this UI is surprisingly simple to use as follows:

ui.Container(uid: "GoogleBooksList", clocator: [tag: "table", id: "hp_table"]) {
	List(uid: "subcategory", clocator: [tag: "td", class: "sidebar"], separator:
	"div") {
	Container(uid: "{all}") {
		TextBox(uid: "title", clocator: [tag: "div", class: "sub_cat_title"])
		List(uid: "links", separator: "p") {
			UrlLink(uid: "{all}", clocator: [:])
		        }
		}
        }}

The Tellurium UID description language (UDL) provides more flexibility to define dynamic web content. Let us see a complex example.

ui.StandardTable(uid: "GT", clocator: [id: "xyz"], ht: "tbody"){
	TextBox(uid: "{header: first} as One", clocator: [tag: "th", text: "one"], self:
	true)
	TextBox(uid: "{header: 2} as Two", clocator: [tag: "th", text: "two"], self: true)
	TextBox(uid: "{header: last} as Three", clocator: [tag: "th", text: "three"],
	self: true)
	TextBox(uid: "{row: 1, column -> One} as A", clocator: [tag: "div", class: "abc"])
	Container(uid: "{row: 1, column -> Two} as B"){
		InputBox(uid: "Input", clocator: [tag: "input", class: "123"])
		Container(uid: "Some", clocator: [tag: "div", class: "someclass"]){
			Span(uid: "Span", clocator: [tag: "span", class: "x"])
			UrlLink(uid: "Link", clocator: [:])
			}
		   }
		TextBox(uid: "{row: 1, column -> Three} as Hello", clocator: [tag: "td"], self:
		true)
	}

In the above example, we use meta data "first", number, and "last" to indicate the header positions. The meta data "{row: 1, column -> One} as A" means the UI element, a TextBox in our case, is in row 1 and the same column as where the header "One" is. The test code is very clean, for example:

getText "GT.A"
keyType "GT.B.Input", input
click "GT.B.Some.Link"
waitForPageToLoad 30000

Future Plans

Tellurium is a young and innovative framework with many novel ideas from both the development team and the user community.  There are many areas Tellurium would like to develop:

1. Tellurium 0.7.0 has implemented a new test driving engine using jQuery.  The main features of the Engine are: UI module group locating, UI module caching, Command bundle processing, Selenium APIs re-implemented in jQuery, and new Tellurium APIs.  Tellurium will continue to develop the new Engine to its maturity.
2. Tellurium UI module plugin 0.8.0 RC1 is just released and it includes many new features. Tellurium IDE release candidate is also out to record and generate test scripts. They are key to the success of Tellurium and they will continue to be improved upon. Besides Trump and Tellurium IDE, Tellurium is planning to improve TelluriumWorks so that it will edit, complete syntax checks, and run Tellurium DSL test scripts.
3. Tellurium as a cloud testing tool is another very important future development. The project team is planning to rethink the architecture to make it more straightforward to execute tests in parallel.  It is very challenging to exploit peer-to-peer techniques to make the test servers capable of self-organizing and self-coordinating in the cloud environment with the least management effort.

Other areas of the framework to be developed include:

• The creation of reusable Dojo, ExtJS, and jQuery UI Tellurium widgets. This will allow other people to reuse the widgets simply by including the jar files in their projects.
• Behavior Driven Testing support.
• Testing flow support. 
• Web security testing.
• Support for other languages such as Ruby.

About The Author

Jian Fang graduated from Georgia Institute of Technology with a Ph.D. degree in Electrical and Computer Engineering. He works as a senior software engineer in a company in the IT industry and mainly focuses on the design and implementation of enterprise applications. He is the creator of the Tellurium Automated Testing Framework.

1. Back to top

   tellurium rocks!!!!!

   by Haroon Rasheed

   We have been using Tellurium for some time now, our tests are more expressive now with low cost of maintenance.

   Reply

2. Back to top

   Some Resources for Tellurium

   by John Fang

   Project site:
   code.google.com/p/aost/
   
   Tellurium User Group
   groups.google.com/group/tellurium-users
   
   Tellurium on Twitter
   twitter.com/TelluriumSource
   
   Tellurium IDE
   code.google.com/p/aost/wiki/TelluriumIde080RC1

   Reply

3. Back to top

   Tellurium

   by Jade Lindquist

   We're using tellurium's tests and data-driven tests for our testing. With Tellurium, we've been able to overcome issues that we had with Selenium tests on web pages with Ajax. The Tellurium developers are the most responsive open source developers that I've interacted with. Awesome software!
   
   Jade

   Reply

4. Back to top

   Verbose?

   by Vikas Hazrati

   May be I am jumping the gun here, since I have not given it a try yet or may be I am too used to Selenium but the code examples for Tellurium look pretty verbose syntactically as compared to Selenium

   Reply

5. Back to top

   Re: Verbose?

   by John Fang

   Consider the following command in Selenium:
   
   selenium.click("//input[@value='Google Search' and @type='button']");
   
   In Tellurium, the command is
   
   click "Google.Search"
   
   Which one is verbose?

   Reply

6. Back to top

   Responsive community

   by Jonathan Share

   Not only is Tellurium a great framework for doing web-testing but the community, although small, is very responsive to the needs of the users. I've seen a very quick response to problems I've reported via the mailing list.

   Reply

7. Back to top

   Re: Verbose?

   by Behrang Saeedzadeh

   Compare full working snippets. Your example is misleading. You haven't shown the code necessary to create the "Google" module in Tellurium.

   Reply

8. Back to top

   Re: Tellurium

   by Behrang Saeedzadeh

   we've been able to overcome issues that we had with Selenium tests on web pages with Ajax

   
   
   Could you please elaborate? In particular, what type of Ajax testing issues are you referring to?

   Reply

9. Back to top

   Re: Verbose?

   by John Fang

   I don't think InfoQ wants me to post all code, but most stuff has already been posted. Here is the full UI module class:
   
   public class GoogleSearchModule extends DslContext {
   
   public void defineUi() {
   ui.Container(uid: "GoogleSearchModule", clocator: [tag: "td"]){
   InputBox(uid: "Input", clocator: [title: "Google Search"])
   SubmitButton(uid: "Search", clocator: [name: "btnG", value: "Google Search"])
   SubmitButton(uid: "ImFeelingLucky", clocator: [value: "I'm Feeling Lucky"])
   }
   }
   
   public void doGoogleSearch(String input) {
   type "GoogleSearchModule.Input", input
   click "GoogleSearchModule.Search"
   waitForPageToLoad 30000
   }
   
   }
   
   
   
   

   Compare full working snippets. Your example is misleading. You haven't shown the code necessary to create the "Google" module in Tellurium.

   Reply

10. Back to top

    Re: Verbose?

    by Behrang Saeedzadeh

    And isn't that verbose?

    Reply

11. Back to top

    Re: Verbose?

    by John Fang

    You mean the UI module definition? Isn't it very expressive about
    the UIs under testing? Once you define it, you always use UIDs to
    reference UI elements, which is not verbose.
    
    

    And isn't that verbose?

    Reply

• Older >