BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Writing Automated Tests with Jazz Automation

Posted by Dedrick Boyd on Dec 07, 2013 |

Introduction to Jazz Automation

Jazz Automation is a testing framework built to automate and speed up acceptance/functional testing for all types of web based systems or static websites and in any industry. It also lends itself to easily implement automated integration testing. Historically this type of testing has been all manual, labor intensive, and inaccurate. With automated testing, companies are producing higher quality applications, producing fewer production bugs, and increasing their speed to market. Jazz Automation supports the following web browsers: IE, Firefox, Safari, Chrome.

Jazz Automation differs from other automated testing frameworks by allowing test writers to describe how software features work in plain and understandable English.

Most automated testing frameworks, such as Selenium and Cucumber, require programming in languages like Java and Ruby before tests are executed. Jazz Automation provides an abstraction layer, removing programming requirements from test writers and configurators.

Writing Your First Test Script

Let’s take a look at a specification for testing an eCommerce website.

Feature: Go to the Amazon web site, search for Harry Porter in the book section. Find
 Harry Potter and the Chamber of Secrets (book2) and then add it to the cart.
   
   Background: Establish the test settings for the test
     Given the following settings:
       | url             | http://www.amazon.com | 
       | platform        | Vista                 |
       | browser         | firefox               |
       | browser version | 23                    |
   Scenario: Verify that we have an empty cart on the amazon home page
     Given I am ON the "HomePage"
     Then I should EXPECT
       | cartCount       | 0                     |
   Scenario: Search for Harry Porter from the books category
     Given I am ON "HomePage"
     And I click "allButton"
     And I select
       | selectCategory  | Books                  |
     And I enter
       | searchField     | Harry Potter           | 
     And I click "go"
     And I wait 5 seconds
     Then I should be ON the "SearchResultsPage"
   Scenario: From the Search Results Page, verify the first result Special
             Edition Harry Potter Paperback Box Set 
     Given I am ON "SearchResultsPage"
     Then I should EXPECT
       | firstResult           | Special Edition Harry Potter Paperback Box Set |
       | chamberOfSecretsBook2 | visible
       |
   Scenario: Click on the Chamber Of Secrets Book2 to reveal the details
        Given I am ON "SearchResultsPage" 
        And I CLICK "chamberOfSecretsBook2"
        Then I should be ON the "BookDetailPage"
   Scenario: Verify the details
        Given I am ON the "BookDetailPage" 
        Then I should EXPECT
        | kindlePrice          | $7.99                    |
        | hardcoverPrice       | $13.94                   |
        | paperbackPrice       | $8.56                    |
    Scenario: Add the book to my cart
      Given I am ON the "BookDetailPage"
      And I click "hardcover"
      And I click "addToCart"
      Then I should be ON the "PreCheckoutPage" 
    Scenario: On the pre-checkout page, verify the details
      Given I am ON the "PreCheckoutPage"
      Then I should EXPECT
        | orderSubtotal            | 13.94       |
        | cartCount                | 1           |
    Scenario: proceed to checkout
      Given I am ON the "PreCheckoutPage"
         And I click "proceedToCheckout"
         Then I should be ON the "SignInPage" 
         The test script should be extremely easy to read and understand, right? 

The test case language shown above is Gherkin. Gherkin is a domain specific language (DSL) that allows us to describe how an application should work in plain, understandable English. When developing test scripts, the scripter should not need to understand or describe the implementation. Most of the above specification is free-text; however Gherkin has a few key words: Feature, Scenario, Given, When, And, and Then. Jazz Automation has extended this list of key words to include: Expect, On, Click, Hover, Wait, Refresh, Enter, Select, Forward, Backward, Visible, and Invisible.

There’s more to Gherkin than will be discussed here; however what’s important to know is that it uses a line-based format must remain intact, so that Jazz Automation will know how to execute the steps for each feature.

With Jazz Automation, there is no need to write any code, be that Java, Ruby, etc. to execute the above test. With Jazz Automation, each website page must be defined as a Page. Both JSON and XML are supported for Page configuration. Each page contains high level attributes that allow Jazz Automation to verify that a page has been loaded properly. Also, each page should have one or more elements defined. Take a closer look at the above specification and you will notice that several words are surrounded by double quotes. The objects within the double quotes are significant to Jazz Automation. These represent Page objects or elements on a page. These elements are defined in a one or more page files for each project.

Let’s take a look at the test folder structure.

First we notice the /features directory. This where all test specifications should reside. The file is of type .feature. Jazz Automation can execute one or more features at a time. The features and order in which they are executed is defined in the jazz.properties file. Next, we notice there are several ‘page files’ located in the /pages directory. This is where a Test Configurator will define each page and page elements that will be accessed during a test. The test specifications can only refer to pages defined here and to page elements defined within a page file. We will cover custom locations for page files later. Lastly, we notice the mandatory jazz.properties file. Let’s skip over it for now.

On to Page Configuration

Test Configurators are the key to properly configuring a Jazz Automation test. Jazz Automation supports both JSON and XML for page configuration.

Test configurators need to have experience with some web technologies such as jQuery. Test configurators need to create a page for each website page to be accessed.

A page has a few required attributes. Each page must have a pageName and at least one domElement. The pageName is heavily used in features and it should be meaningful so that testers know exactly website page it refers to. DomElements are the web page elements used to validate the software. Typically, they refer to label values, drop-down values, images, buttons, data displayed (pricing, etc.). The name of these elements should be meaningful so that the test writers understand each component’s purpose.

Let’s take a look at a JSON page configuration.

{
    "pageName":"HomePage",
    "title": "Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more",
    "urlExtension": "www.amazon.com",
      "keyDomElementName": "searchField",
    "domElements" :
    {
        "cartCount" :
        {
            "jquery": "$('#nav-cart-count')"
        },
        "allButton" :
        {
            "jquery": "$('#nav-search-in')"
        },
        "selectCategory" :
        {
            "jquery": "$('#searchDropdownBox')"
        },
        "searchField" :
        {
            "jquery": "$('#twotabsearchtextbox')"
        },
        "go" :
        {
            "jquery": "$('.nav-submit-input')"
        }
    }
}

Let’s define the key components of a page configuration file.

Component

Required

Blank Allowed

Omission Allowed

Notes

pageName

Yes

No

No

By convention, the value should match the file name.

Title

No

Yes

Yes

 

urlExtension

No

Yes

No

 

keyDomElementName

No

Yes

Yes

Recommended to ensure the page has loaded fully, prior to moving forward with the scenario

domElements

Yes

No

No

 

That’s it!

Once the remaining pages are defined, we can run the scenario through completion. Test configurators are required to know one of the following: jQuery, XPath, or CSS. We use these to ‘select’ the web elements needed for testing. JQuery is the preferred selector mechanism for Jazz Automation, and most QA analysts can pick up jQuery in a few days, especially with all the online tutorials and modern-day web browser developer tools available.

General Configuration

The final item for discussion is the required jazz.properties file. This configuration file contains a few general configuration items such as the project name (used in reporting), a flag indicating if the tests should be run remotely or on the local system, action and page pace, and the features to be executed. There are additional optional configuration properties that have been omitted.

#project name
projectName: Amazon

# turn on remote support. if useRemote true, you must specify the following remoteWebDriverUrl property.
useRemote: false

# remote web driver url - usually your sauceLabs url
remoteWebDriverUrl: http://jazzautomation:****@ondemand.saucelabs.com:80/wd/hub

# you can customize you folder name of pages. By default it is pages
#pagesDirectoryName: pages

# page pacing between two pages loading
pagePace: 10
#action pacing between two actions on the same page. The default value is "1 second". 
actionPace: 1

# features: can be a list of features, separated by comma
features:searchForHarryPotter

Reporting

For each test, Jazz Automation produces a report that details the test run. If the tests are successful, a simple report is generated with a success rate of 100%. Upon any failure, a screen shot will be taken where the error occurred and inserted into the report. Additionally, the report is generated from a data file produced during the test run. This data file can be used for historical tracking, charting, and trending.

Let’s take a look at a failed test report:

(Click on the image to enlarge it)

Conclusion

This concludes our tour of writing automated tests using Jazz Automation. For the curious, there’s plenty more to learn. For example, Jazz Automation can also be used remotely with technology offered by companies such as Sauce Labs so that your organization does not have to maintain every possible browser and operating system combination required for full scale testing. One can also dig into how Jazz Automation works with websites that do not use jQuery. Hint: it’s dynamically inserted.

Jazz Automation completes the continuous integration loop by also testing the user interface and providing immediate feedback. This is more critical than ever as the industry moves to feature rich HTML5 and JavaScript user interfaces. Organizations need to know immediately that the application is working as expected from top to bottom. Further, the use of Gherkin as our DSL will allow key stakeholders to articulate their understanding of how the application should work and communicated effectively with technical resources, while leaving the development team free to build the product.

Jazz Automation should not be confused as a replacement for conventional unit and integration testing frameworks such as JUnit or TestNG. Unit and integration testing is still critical to successful software implementation; Jazz Automation takes it a step further and now allows automated tests to be written against the user interface. Additionally, Jazz Automation tests at the DOM level, so you can be sure that moving items around within a web page will not result in test errors.

Automated user interface testing is not a priority for every organization, but where it is, never let front-end or regression testing slow you down again.

About the Author

Dedrick Boyd is a seasoned IT professional with extensive experience across the full software development and product life cycles. He has technical skills and management experience with small and large development teams in a variety of industries including retail, pharmaceutical, media and entertainment, healthcare, and financial services. Dedrick is currently employed at Statement, where he is a technology innovator as well as a relationship manager. He holds a bachelors in Management Information Systems from Florida State University. Dedrick’s hobbies include world travel, reading, cooking, american football, and spending time with his family.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Been there, done that, got the T-Shirt by Aslak Hellesøy

The people behind Jazz Automation seem to reimplement all the features we removed from Cucumber because we came to the conclusion that they were terrible.

You can read more about that here: aslakhellesoy.com/post/11055981222/the-training...

And here: skillsmatter.com/podcast/agile-testing/refuctor... (video)

Good riddance

Re: Been there, done that, got the T-Shirt by Dedrick Boyd

Thanks for your reply Aslak. It appears to me that the tool that you are promoting and our tool are aimed at vastly different user bases. Jazz Automation was built with a specific design in mind, zero programming required. Our Fortune 100 and 500 clients have QA organizations that often do not have programming experience and they are very interested in testing their web applications as part of the continuous integration cycle. They are also not interested in diverting their development resources towards this type of test automation. That is why we have had great success with Jazz Automation.

Can I run Jazz Automation from Maven? by Matt Raible

Is it possible to run Jazz Automation from Maven? Or at least configure it to run in my CI server?

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT