Twist 2.0 Supports Behavior Driven and Collaborative Testing

ThoughtWorks Studios recently released the latest version and second major revision of Twist, its Agile test automation tool. Twist 2.0 helps testers, developers and business analysts with collaborative testing. It uses Behavior Driven Development (BDD) and Domain Specific Language (DSL) techniques to allow test automation and also supports writing tests scripts using Groovy dynamic language.

Twist offers an user interface to create reusable test automation scripts for both manual and functional automated tests. It also provides continuous visibility into testing activities across distributed projects and teams. The features in the latest version include:

• Hybrid Test Execution: This allows for manual and automated test steps within the same test scenario to improve collaboration of both automated and manual testing efforts.
• Test Scenario Reuse: It provides high level test scenarios and steps (manual and automated) that can be reused and shared within and across multiple projects.
• Sahi Web Driver Support: Testers have the option of using Sahi or Selenium for Web testing.
• SWTBOT Support: allows Twist users to test Java SWT-based applications
• Data Driven Scenarios: This allows to build data-driven tests on the fly for increased test coverage

InfoQ spoke with Chad Wathington of ThoughtWorks Studios team to get more details on the new release of their test automation tool.

InfoQ: What was the main motivation for the new release of Twist product?

With Twist 2.0 we're continuing to add the features necessary to help teams be successful with large-scale functional test automation. We've made it easier to write data driven tests. We've added Groovy support to make writing test mechanics faster and easier. We've made it a breeze to add new layers of abstraction to your tests. We added the ability to run tests that have manual and automated steps to help users with exploratory testing. Ultimately, we're continuing on our mission to give testers, business users, and developers the right tool support to build, collaborate on, and maintain large test suites.

InfoQ: Can explain more about the business users writing out scenarios to express requirements, like what syntax they need to follow, how does the tool help them in documenting the requirements (via code completion) etc?

We envision that at a minimum, business users should be able to read Twist scenarios easily. For more advanced users, who understand a little logic and how test scenarios should be created, they can use the editor to construct tests using a functional testing DSL (domain specific language). Twist also supports manual tests, so less technical users can create a manual test first, and technical users can turn it into an automated test when the team is ready.

The Twist Scenario Editor is essentially a rich text editor that gives business users a lot of flexibility to write whatever is contextually relevant for a test. The executable parts of a scenario, where you can build the actual DSL, are written in plain English (or almost any other language), so there's virtually no special syntax to learn. We purposely chose to be more flexible than many of the ATDD and BDD frameworks out there, so there are no magic words or imposed sentence structure.

Control-Space gives you auto-complete/contextual assistance wherever you are in the app, so it's a nice way to reuse parts of your DSL. Lastly, any word inside double quotes is a parameter, so there's no funky variable syntax. It's that easy.

InfoQ: How does Twist compare with other BDD frameworks such as JBehave, easyb, or Cucumber?

Twist has a couple of key differences. It's not a framework so much as an IDE -- it's built around the idea of giving testers, business people, and developers testing specific IDE support. Twist helps you work with test suites, not just individual tests, and ultimately managing a suite is the toughest problem down the line. So, we've made it very convenient to figure out what's in the DSL you've created via auto-complete. We've made it very easy to create additional layers of abstraction in the semantics of your tests, not just in the code. We've made it very easy to refactor your test suite, and we've given you the ability to mix manual and automated testing. Most BDD tools are developer tools that have trickled to testers. We've built a tool for testers, devs, and business users.

InfoQ: How does the Data Driven Test Scenario feature in Twist 2.0 compare with other data testing frameworks like DBUnit?

We've started at a different level. DBUnit is about maintaining database integrity across test runs (and in fact you can use DBUnit as part of the set up and tear down of your tests in Twist). We are trying to give people a way to drive their testing scenarios via a large data set. Say for example you've written the following test in Twist:

Merge Bank Accounts on Login:

On the Login Page:

• Login as "johndoe" with password "abc123"

On Account Merge Page:

• When presented with merge prompt, "accept" it
• Choose username "johndoe" as primary account
• Accept license terms
• Verify that you have successfully changed merged your accounts

After you've written this test, you might want to try a bunch of different combinations, given that the parameters are the words in quotes. You can right-click on this test and automatically create a data table that will run n times with a specified data set. Twist manages the parameters for you so that your tests are still readable - there are no variable names in your scenario - just a clean table. We view this as putting semantic meaning and intent around what would have been a plain old column fixture in something like FIT.

InfoQ: What type of modern programming features does Groovy Scripting feature make available to the developers?

Because Groovy is a dynamic language, it can be substantially more terse and readable. Obviously, Groovy's first class support for closures makes some tasks that are onerous in Java a breeze. Iterating over collections is a stock example. Closures can also make code more readable as well. For example with(), which is on object in Groovy, makes Selenium's flat API look cleaner. Instead of the following in Java:

selenium.open("http://foo.com");
selenium.click("link=foobar");
assertTrue (selenium.isTextPresent("myfoo"));

In Groovy, you have:

selenium.with {
open "http://foo.com"
click "link=foobar"
assertTrue isTextPresent("myfoo")
}

• Groovy's standard library packages go beyond what's offered in Java. For example groovy.sql makes working with DBs simpler than stock JDBC in many instances. It's also great to have access to anything written in Java.
• You can also use metaprogramming techniques. For example, you can monkey patch the Selenium class on the fly if you need to add new functionality, which is a lot faster than writing a new subclass and rewiring up Spring, which injects drivers into Twist scenarios by default.

InfoQ: Can you elaborate on the following statement from the product announcement and explain how the tool helps in not having to rewrite the test scripts: "Simplified maintenance - Changes in application do not require rewrite of test scripts."

Twist is designed around the idea that well-abstracted test scenarios, which happen to be both semantically meaningful and declarative, can help you with test maintenance. So if you write your tests as click this button, type this text in a widget, and check this box, which is what people typically do in a standard driver framework like Selenium, then that's difficult to maintain. But if you say, add widget x to my shopping cart, add my billing address "abc", and checkout, without regard to how you do those things, then you have the building blocks for a modular test suite. When the underlying mechanics of those actions change, you can change them in one place. We can make testing GUI apps a lot less brittle.

Another key part is refactoring. Twist supports two-way refactoring between your DSL-based test scenarios and your code (with the exception of one case in Groovy). So essentially, you can make changes across your entire test suite pretty easily. Say for example, you want to change the concept of login in your application from just using username (which was where you started at in story 1), to username and password. You can rephrase login to take both username and password in one of your scenarios, and it will globally apply, even for the code. At the code level, you could instead use an Introduce Parameter refactoring, and your test scenarios would likewise reflect the new parameter. Or, say you want to introduce OpenID. You can overload login at the code level, and then your scenarios will accept either username and password or open ID. You can essentially make large changes without having to rewrite several tests.

InfoQ: Can Twist be used as a stand-alone product with other CI tools like Hudson or Subversion?

Yes. Twist scenarios are plain text, so they can be checked into any source control management system. Twist also will run in any CI environment (and we ship with a custom Ant task), and its reports are JUnit compliant, so your standard build and CI tools can parse them.

InfoQ: What is the future road-map of Twist product in terms of new features and enhancements?

We're going to continue to improve the Scenario Editor, making the test creation and maintenance process simpler and richer. We're going make it even easier to do exploratory testing. We're also going to make test management easier as well, as well as improve the integration with our other products Mingle and Cruise. We have a lot more in the roadmap, but that's all we're saying for now.