BT

Beyond Page Objects: Next Generation Test Automation with Serenity and the Screenplay Pattern

Posted by John Ferguson Smart, Antony Marcano, Andy Palmer, Jan Molak on Mar 15, 2016 |

Automated acceptance testing is essential in today's fast-paced software delivery world. A high quality set of automated acceptance tests helps you deliver valuable features sooner by reducing the wasted time spent in manual testing and fixing bugs. When combined with Behaviour-Driven Development, automated acceptance testing can guide and validate development effort, and help teams focus both on building the features that really matter and ensuring that they work.

But automated acceptance testing is not easy; like any other software development activity, it requires skill, practice and discipline. Over time, even teams with the best intent can see their test suites become slow, fragile and unreliable. It becomes increasingly more difficult to add new tests to the existing suite; teams lose confidence in the automated tests, compromising the investment in the test suite, and affecting team morale. We routinely see even experienced teams, using design patterns such as Page Objects, running into this sort of issue. Page Objects are a good place to start for teams not familiar with patterns and design principles used by proficient programmers (e.g. SOLID) but the importance of bringing strong technical skills to the team should be considered early on in a project to avoid these challenges.

The Screenplay Pattern (formerly known as the Journey Pattern) is the application of SOLID design principles to automated acceptance testing, and helps teams address these issues. It is essentially what would result from the merciless refactoring of Page Objects using SOLID design principles. It was first devised by Antony Marcano between 2007 - 2008, refined with thinking from Andy Palmer from 2009. It didn’t receive the name "the Journey Pattern" until Jan Molak started working with it in 2013. Several people have written about it under this name already, however, the authors now refer to it as the Screenplay Pattern.

The Screenplay Pattern is an approach to writing high quality automated acceptance tests based on good software engineering principles such as the Single Responsibility Principle, and the Open-Closed Principle. It favours composition over inheritance, and employs thinking from Domain Driven Design to reflect the domain of performing acceptance tests, steering you towards the effective use of layers of abstraction. It encourages good testing habits and well-designed test suites that are easy to read, easy to maintain and easy to extend, enabling teams to write more robust and more reliable automated tests more effectively.

Serenity BDD is an open source library designed to help you write better, more effective automated acceptance tests, and use these acceptance tests to produce high quality test reports and living documentation. As we will see in this article, Serenity BDD has strong built-in support for using the Screenplay Pattern straight out of the box.

The Screenplay Pattern in Action

In the rest of this article, we will be using Serenity BDD to illustrate the Screenplay Pattern, though the pattern itself is largely language and framework-agnostic. The application we will be testing is the AngularJS implementation of the well-known TodoMVC project (see Figure 1).

Figure 1 The Todo application

For simplicity, we will be using Serenity BDD with JUnit, though we could also choose to implement our automated acceptance criteria using Serenity BDD with Cucumber-JVM or JBehave.

Now suppose we are implementing the “Add new todo items” feature. This feature could have an acceptance criterion along the lines of “Add a new todo item”. If we were testing these scenarios manually, it might look like this:

  • Add a new todo item
    • Start with an empty todo list
    • Add an item called ‘Buy some milk’
    • The ‘Buy some milk’ item should appear in the todo list

One of the big selling points of the Screenplay Pattern is that it lets you build up a readable API of methods and objects to express your acceptance criteria in business terms. For example, using the Screenplay Pattern, we could automate the scenario shown above very naturally like this:

givenThat(james).wasAbleTo(Start.withAnEmptyTodoList());
when(james).attemptsTo(AddATodoItem.called("Buy some milk"));
then(james).should(seeThat(TheItems.displayed(), hasItem("Buy some milk")));

If you have used Hamcrest matchers, this model will be familiar to you. When we use a Hamcrest matcher we are creating an instance of a matcher that will be evaluated in an assertThat method. Similarly, AddATodoItem.called() returns an instance of a ‘Task’ that is evaluated later in the attemptsTo() method. Even if you are not familiar with how this code is implemented under the hood, it should be quite obvious what the test is trying to demonstrate, and how it is going about it.

We will see soon how writing this kind of test code is as easy as reading it.

Declarative code written in a way that reads like business language is significantly more maintainable and less prone to error than code written in a more imperative, implementation-focused way. If the code reads like a description of the business rules, it is a lot harder for errors in business logic to slip into the test code or into the application code itself.

Furthermore, the test reports generated by Serenity for this test also reflect this narrative structure, making it easier for testers, business analysts and business people to understand what the tests are actually demonstrating in business terms (see Figure 2).

Figure 2: This Serenity report documents both the intent and the implementation of the test

The code listed above certainly reads cleanly, but it may leave you wondering how it actually works under the hood. Let’s see how it all fits together.

Screenplay Pattern tests run like any other Serenity test

At the time of writing, the Serenity Screenplay implementation integrates with both JUnit and Cucumber. For example, in JUnit, you use the SerenityRunner JUnit runner, as for any other Serenity JUnit tests. The full source code of the test we saw earlier is shown here, where an “Actor” plays the role of a user interacting with the system:

@RunWith(SerenityRunner.class)
public class AddNewTodos {

    Actor james = Actor.named("James");

    @Managed private WebDriver hisBrowser;

    @Before
    public void jamesCanBrowseTheWeb() {
        james.can(BrowseTheWeb.with(hisBrowser));
    }

    @Test
    public void should_be_able_to_add_a_todo_item() {

        givenThat(james).wasAbleTo(Start.withAnEmptyTodoList());

        when(james).attemptsTo(AddATodoItem.called("Buy some milk"));

        then(james).should(seeThat(TheItems.displayed(), 
                                    hasItem("Buy some milk")));
    }
}

It’s not hard to glean what this test does just by reading the code. There are however a few things here that will be unfamiliar, even if you have used Serenity before. In the following sections, we will take a closer look at the details.

The Screenplay Pattern encourages strong layers of abstraction

Experienced automated testers use layers of abstraction to separate the intent of the test (what you are trying to achieve) from the implementation details (how you achieve it). By separating the what from the how, the intent from the implementation, layers of abstraction help make tests easier to understand and to maintain. Indeed, well defined layers of abstraction are perhaps the single most important factor in writing high quality automated tests.

In User Experience (UX) Design, we break down the way a user interacts with an application into goals, tasks and actions:

  • The goal describes the ‘why’ of the scenario in terms of what the user is trying to achieve in business terms.
  • The tasks describe what the user will do as high-level steps required to achieve this goal.
  • The actions say how a user interacts with the system to perform a particular task, such as by clicking on a button or entering a value into a field.

As we will see, the Screenplay Pattern provides a clear distinction between goals (scenario titles), tasks (top-level of abstraction in the scenario) and actions (the lowest level of abstraction, below the tasks), which makes it easier for teams to write layered tests more consistently.

The Screenplay Pattern uses an actor-centric model

Tests describe how a user interacts with the application to achieve a goal. For this reason, tests read much better if they are presented from the point of view of the user (rather than from the point of ‘pages’).

In the Screenplay Pattern, we call a user interacting with the system an Actor. Actors are at the heart of the Screenplay pattern (see Figure 3). Each actor has one or more Abilities, such as the ability to browse the web or to query a restful web service. Actors can also perform Tasks such as adding an item to the Todo list. To achieve these tasks, they will typically need to interact with the application, such as by entering a value into a field or by clicking on a button. We call these interactions Actions. Actors can also ask Questions about the state of the application, such as by reading the value of a field on the screen or by querying a web service.

Figure 3: The Screenplay Pattern uses an actor-centric model

In Serenity, creating an actor is as simple as creating an instance of the Actor class and providing a name:

Actor james = Actor.named("James");

We find it useful to give the actors real names, rather than use a generic one such as “the user”. Different names can be a shorthand for different user roles or personas, and make the scenarios easier to relate to. For more information on using Personas, see Jeff Patton’s talk “Pragmatic Personas”

Actors have abilities

Actors need to be able to do things to perform their assigned tasks. So we give our actors “abilities”, a bit like the superpowers of a super-hero, if somewhat more mundane. If this is a web test, for example, we need James to be able to browse the web using a browser.

Serenity BDD plays well with Selenium WebDriver, and is happy to manage the browser lifecycle for you. All you need to do is to use the @Managed annotation with a WebDriver member variable, as shown here:

@Managed private WebDriver hisBrowser;

We can then let James use this browser as follows:

james.can(BrowseTheWeb.with(hisBrowser));

To make it clear that this is a precondition for the test (and could very well go in a JUnit @Before method), we can use the syntactic sugar method givenThat():

givenThat(james).can(BrowseTheWeb.with(hisBrowser));

Each of the actor’s abilities is represented by an Ability class (in this case BrowseTheWeb) that keeps track of the things the actor requires in order to perform this ability (for example, the WebDriver instance used to interact with the browser). Keeping the things an actor can do (browse the web, invoke a web service…) separate from the actor makes it easier to extend the actor’s abilities. For example, to add a new custom ability, you just need to add a new Ability class to your test classes.

Actors perform tasks

An actor needs to perform a number of tasks to achieve a business goal. A fairly typical example of a task is “adding a todo item”, which we could write as follows:

james.attemptsTo(AddATodoItem.called("Buy some milk"))

Or, if the task is a precondition, rather than the main subject of the test, we could write something like this:

james.wasAbleTo(AddATodoItem.called("Buy some milk"))

Let’s break it down to understand what is going on. At the heart of the Screenplay Pattern, an actor performs a sequence of tasks. In Serenity, this mechanism is implemented in the Actor class using a variation of the Command Pattern, where the actor executes each task by invoking a special method called performAs() on the corresponding Task object (see Figure 4):

Figure 4: The actor invokes the performAs() method on a sequence of tasks

Tasks are just objects that implement the Task interface, and need to implement the performAs(actor) method. In fact, you can think of any Task class as basically a performAs() method alongside a supporting cast of helper methods.

Tasks can be created using annotated fields or builders

To do its reporting magic, Serenity BDD needs to instrument the task and action objects used during the tests. The simplest way to do arrange this is to let Serenity create it for you, just like any other Serenity step library, using the @Steps annotation. In the following code snippet, Serenity will instantiate the openTheApplication field for you, so that James can use it to open the application:

@Steps private OpenTheApplication openTheApplication;
…
james.attemptsTo(openTheApplication);

This works well for very simple tasks or actions, for example ones that take no parameters. But for more sophisticated tasks or actions, a factory or builder pattern (like the one used with our earlier AddATodoItem) is more convenient. Experienced practitioners generally like to make the builder method and the class name combine to read like an English sentence, so that the intent of the task remains crystal clear:

AddATodoItem.called("Buy some milk")

Serenity BDD provides the special Instrumented class that makes it easy to create task or action objects using the builder pattern. For example, the AddATodoItem class has an immutable field called thingToDo, that contains the text to go in the new Todo item.

public class AddATodoItem implements Task {

    private final String thingToDo;

    protected AddATodoItem(String thingToDo) { this.thingToDo = thingToDo; }
}

We can invoke this constructor using the Instrumented.instanceOf().withProperties() methods, as shown here:

public class AddATodoItem implements Task {

    private final String thingToDo;

    protected AddATodoItem(String thingToDo) { this.thingToDo = thingToDo; }

    public static AddATodoItem called(String thingToDo) {			
        return Instrumented.instanceOf(AddATodoItem.class).
                                                  withProperties(thingToDo);
    }
}

High-level tasks are composed of other lower-level tasks or actions

To get the job done, a high-level business task will usually need to call either lower level business tasks or actions that interact more directly with the application. In practice, this means that the performAs() method of a task typically executes other, lower level tasks or interacts with the application in some other way. For example, adding a todo item requires two UI actions:

  1. Enter the todo text in the text field
  2. Press Return

The performAs() method in the AddATodoItem class used earlier does exactly that:

    private final String thingToDo;

    @Step("{0} adds a todo item called #thingToDo")
    public <T extends Actor> void performAs(T actor) {			
        actor.attemptsTo(					
                          Enter.theValue(thingToDo)
                              .into(NewTodoForm.NEW_TODO_FIELD)
                              .thenHit(RETURN)
        );
    }

The actual implementation uses the Enter class, a pre-defined Action class that comes with Serenity. Action classes are very similar to Task classes, except that they focus on interacting directly with the application. Serenity provides a set of basic Action classes for core UI interactions such as entering field values, clicking on elements, or selecting values from drop-down lists. In practice, these provide a convenient and readable DSL that lets you describe common low-level UI interactions needed to perform a task.

In the Serenity Screenplay implementation, we use a special Target class to identify elements using (by default) either CSS or XPATH. The Target object associates a WebDriver selector with a human-readable label that appears in the test reports to make the reports more readable. You define a Target object as shown here:

 Target WHAT_NEEDS_TO_BE_DONE = Target.the(
                "'What needs to be done?' field").locatedBy("#new-todo")
;

Targets are often stored in small Page-Object like classes that are responsible for one thing, knowing how to locate the elements for a particular UI component, such as the ToDoList class shown here:

public class ToDoList {
   public static Target WHAT_NEEDS_TO_BE_DONE = Target.the(
        "'What needs to be done?' field").locatedBy("#new-todo");
   public static Target ITEMS = Target.the(
        "List of todo items").locatedBy(".view label");
   public static Target ITEMS_LEFT = Target.the(
        "Count of items left").locatedBy("#todo-count strong");
   public static Target TOGGLE_ALL = Target.the(
        "Toggle all items link").locatedBy("#toggle-all");
   public static Target CLEAR_COMPLETED = Target.the(
        "Clear completed link").locatedBy("#clear-completed");
   public static Target FILTER = Target.the(
        "filter").locatedBy("//*[@id='filters']//a[.='{0}']");
   public static Target SELECTED_FILTER = Target.the(
        "selected filter").locatedBy("#filters li .selected");
}

The @Step annotation on the performAs() method is used to provide information about how the task will appear in the test reports:


    @Step("{0} adds a todo item called #thingToDo")
    public <T extends Actor> void performAs(T actor) {…}

Any member variables can be referred to in the @Step annotation by name using the hash (‘#’) prefix (like “#thingToDo” in the example). You can also reference the actor itself using the special “{0}” placeholder. The end result is a blow-by-blow account of how each business task was performed (see Figure 5).

Figure 5: Test reports show details about both tasks and UI interactions

Tasks can be used as building blocks by other tasks

It is easy to reuse tasks in other, higher level tasks. For example, the sample project uses a AddTodoItems task to add a number of todo items to the list, like this:

givenThat(james).wasAbleTo(AddTodoItems.called("Walk the dog", 
                                               "Put out the garbage"));

This task is defined using the AddATodoItem class, as shown here:

public class AddTodoItems implements Task {

   private final List<String> todos;

   protected AddTodoItems(List<String> items) { 
       this.todos = ImmutableList.copyOf(items); }

   @Step("{0} adds the todo items called #todos")
   public <T extends Actor> void performAs(T actor) {
       todos.forEach(
               todo -> actor.attemptsTo(
                   AddATodoItem.called(todo)
               )
       );
   }

   public static AddTodoItems called(String... items) {
       return Instrumented.instanceOf(AddTodoItems.class).
                              withProperties(asList(items));
   }
}

 

It is quite common to reuse existing tasks to build up more sophisticated business tasks in this way. A convention that we have found useful is to break from the common Java idiom and put the static creation method below the performAs() method. This is because the most valuable information inside a Task is how it is being performed rather than how it is created.

Actors can ask questions about the state of the application

A typical automated acceptance test has three parts:

  1. Set up some test data and/or get the application into a known state
  2. Perform some action
  3. Compare the new application state with what is expected.

From a testing perspective, the third step is where the real value lies – this is where we check that the application does what it is supposed to do.

In a traditional Serenity test, we would write an assertion using a library like Hamcrest or AssertJ to check an outcome against an expected value. Using the Serenity Screenplay implementation, we express assertions using a flexible, fluent API quite similar to the one used for Tasks and Actions. In the test shown above, the assertion looks like this:

then(james).should(seeThat(TheItems.displayed(), hasItem("Buy some milk")));

The structure of this code is illustrated in Figure 6.

Figure 6: A Serenity Screenplay assertion

As you might expect, this code checks a value retrieved from the application (the items displayed on the screen) against an expected value (described by a Hamcrest expression). However, rather than passing an actual value, we pass a Question object. The role of a Question object is to answer a precise question about the state of the application, from the point of view of the actor, and typically using the abilities of the actor to do so.

Questions are rendered in human-readable form in the reports

Another nice thing about the Screenplay assertions is that they appear in a very readable form in the test reports, making the intent of the test clearer and error diagnostics easier. (see Figure 8).

Figure 8: Question objects are rendered in human-readable form in the test report

Actors use their abilities to interact with the system

Let’s see this principle in action in another test. The Todo application has a counter in the bottom left hand corner indicating the remaining number of items (see Figure 7).

Figure 7: The number of remaining items is displayed in the bottom left corner of the list

The test to describe and verify this behavior could look like this:

@Test
public void should_see_the_number_of_todos_decrease_when_an_item_is_completed() 
{

   givenThat(james).wasAbleTo(Start.withATodoListContaining(
                                      "Walk the dog", "Put out the garbage"));

   when(james).attemptsTo(
       CompleteItem.called("Walk the dog")
   );

   then(james).should(seeThat(TheItems.leftCount(), is(1)));
}

The test needs to check that the number of remaining items (as indicated by the “items left” counter) is 1. The corresponding assertion is in the last line of the test:

then(james).should(seeThat(TheItems.leftCount(), is(1)));

The static TheItems.leftCount() method is a simple factory method that returns a new instance of the ItemsLeftCounter class, as shown here:

public class TheItems {
   public static Question<List<String>> displayed() {
       return new DisplayedItems();
   }

   public static Question<Integer> leftToDoCount() {
       return new ItemsLeftCounter();
   }
}

This serves simply to make the code read in a fluent fashion.

The Question object is defined by the ItemsLeftCounter class. This class has one very precise responsibility: to read the number in the remaining item count text displayed at the bottom of the todo list.

Question objects are similar to Task and Action objects. However, instead of the performAs() used for Tasks and Actions, a Question class needs to implement the answeredBy(actor) method, and return a result of a specified type. The ItemsLeftCounter is configured to return an Integer.

public class ItemsLeftCounter implements Question<Integer> {
   @Override
   public Integer answeredBy(Actor actor) {
       return Text.of(TodoCounter.ITEM_COUNT)
                  .viewedBy(actor)
                  .asInteger();
   }
}

The Serenity Screenplay implementation provides a number of low-level UI interaction classes that let you query your web page in a declarative way. In the code above, the answeredBy() method uses the Text interaction class to retrieve the text of the remaining item count and to convert it to an integer.

As shown previously, the location logic has been refactored into the TodoList class:

public static Target ITEMS_LEFT = Target.the("Count of items left").
                                          locatedBy("#todo-count strong");

Once again, this code works at three levels, each with distinct responsibilities:

  • The top level step makes an assertion about the state of the application:
    then(james).should(seeThat(TheItems.leftCount(), is(1)));
  • The ItemsLeftCounter Question class queries the state of the application and provides the result in the form expected by the assertion;
  • The TodoList class stores the location of web elements used by the Question class.

Writing custom UI interactions

The Serenity Screenplay implementation comes with a range of low-level UI interaction classes. There may be very rare cases where these don’t meet your needs. In this case, it is possible to interact directly with the WebDriver API. You do this by writing your own Action class, which is easy to do.

For example, suppose we want to delete an item in the todo list, using code along the following lines:

when(james).attemptsTo(
       DeleteAnItem.called("Walk the dog")
);

Now, for reasons related to the implementation of the application, the Delete button does not accept a normal WebDriver click, and we need to invoke the JavaScript event directly. You can see the full class in the sample code, but the performAs() method of the DeleteAnItem task uses a custom Action class called JSClick to trigger the JavaScript event:

@Step("{0} deletes the item '#itemName'")
   public <T extends Actor> void performAs(T theActor) {
       Target deleteButton = TodoListItem.DELETE_ITEM_BUTTON.of(itemName);
       theActor.attemptsTo(JSClick.on(deleteButton));
   }

The JSClick class is a simple implementation of the Action interface, and looks like this:

public class JSClick implements Action {

   private final Target target;

   @Override
   @Step("{0} clicks on #target")
   public <T extends Actor> void performAs(T theActor) {
       WebElement targetElement = target.resolveFor(theActor);
       BrowseTheWeb.as(theActor).evaluateJavascript(
                                   "arguments[0].click()", targetElement);
   }

   public static Action on(Target target) {
       return instrumented(JSClick.class, target);
   }
  public JSClick(Target target) {
       this.target = target;
   }


}

The important code here is in the performAs() method, where we use the BrowseTheWeb class to access the actor’s Ability to use a browser. This gives full access to the Serenity WebDriver API:

BrowseTheWeb.as(theActor).
           evaluateJavascript("arguments[0].click()", targetElement);

(Note that this is a contrived example – Serenity already provides an interaction class to inject Javascript into the page as well).

Page Objects become smaller and more specialized

An interesting consequence of using the Screenplay pattern is that it changes the way you use and think about Page Objects. The idea of a Page Object is to encapsulate the UI-related logic that accesses or queries a web page, or a component on a web page, behind a more business-friendly API. As a concept, this is fine.

But the problem with Page Objects (and with traditional Serenity step libraries, for that matter) is it can be hard to keep them well organized. They tend to grow, becoming bigger and harder to maintain as the test suite grows. This should be no surprise since such page objects violate both the Single Responsibility Principle (SRP) and Open-Closed Principle (OCP) – the ‘S’ and the ‘O’ in SOLID. Many test suites end up with complex hierarchies of Page Objects, inheriting “common” behavior such as menu bars or logout buttons from a parent Page Object, which violates the principle of favoring composition over inheritance. New tests typically need modifications to existing Page Object classes, introducing the risk of bugs.

When you use the Screenplay Pattern, your Page Objects tend to become smaller and more focused, with a clearly defined mandate of locating elements for a particular component on the screen. Once written, they tend to remain unchanged unless the underlying web interface changes.

BDD Style scenarios are not mandatory

Some people writing acceptance tests in an xUnit framework may not like the Given/When/Then style of writing scenarios. These methods are there purely for readability, helping you make your intent explicit by expressing where you arrange (given), act (when) and assert (then). Not everyone likes this style, and so you are not restricted to it. As an alternative you can write:


        james.wasAbleTo(Start.withAnEmptyTodoList());

        james.attemptsTo(AddATodoItem.called("Buy some milk"));

        james.should(seeThat(toDoItems, hasItem("Buy some milk")));

The intent is implicit in the ‘wasAbleTo’, ‘attemptsTo’ and ‘should’ methods, however we believe that making our intent explicit will benefit us and anyone else who will read our code later, and so we would recommend using the built in givenThat(), when(), then() methods. If you are using this approach in Cucumber, you can leave out the Given/When/Then methods as the intent is generally explicit in the Cucumber step definitions.

Conclusion

The Screenplay Pattern is an approach to writing automated acceptance tests founded on good software engineering principles that makes it easier to write clean, readable, scalable, and highly maintainable test code. It is one possibile outcome of the merciless refactoring of the Page Object pattern towards SOLID principles. The new support for the Screenplay Pattern in Serenity BDD opens a number of exciting possibilities. In particular:

  • The declarative writing style encouraged by the Screenplay Pattern makes it much simpler to write code that is easier to understand and maintain;
  • Task, Action and Question classes tend to be more flexible, reusable and readable than traditional Serenity step methods;
  • Separating the abilities of an actor adds a great deal of flexibility. For example, it is very easy to write a test with several actors using different browser instances.

Like many good software development practices, the Screenplay Pattern takes some discipline to start with. Some care is initially required to design a readable, DSL-like API made up of well-organised tasks, actions and questions. However, the benefits become apparent quite quickly when the test suite scales, and these libraries of reusable components help accelerate the test writing process to a sustainable rate, reducing the friction usually associated with the ongoing maintenance of automated test suites.

Further reading

This is just an introduction to the Screenplay Pattern and its implementation in Serenity. The best way to learn more is to study working code. The source code for the sample project can be found on Github.

References

About the Authors

John Ferguson Smart is an experienced author, speaker and trainer specialising in Agile Delivery Practices, currently based in London. An international speaker well known in the Agile community for his many published articles and presentations, particularly in areas such as BDD, TDD, test automation, software craftsmanship and team collaboration, John helps organisations and teams around the world deliver better software sooner and more effectively both through more effective collaboration and communication techniques, and through better technical practices. LinkedInGithubWebsite

Antony Marcano is best known in the community for his thinking on BDD, user stories, testing and writing fluent APIs & DSLs in Ruby, Java and more. He has 16+ years of experience on agile projects and transformations of all shapes and sizes, spending as much of his time as a practitioner as he does as a coach or trainer. He shares his experiences with the community in various ways, including as a contributor to books such as Agile Coaching and Agile Testing with references in Bridging the Communication Gap and Software Craftsmanship Apprenticeship Patterns. His work in the community continues as a regular speaker on agile development at international conferences and as a regular guest speaker at Oxford University.

Andy Palmer is co-creator of the ground-breaking PairWith.us screencasts, regular speaker at international conferences; Andy has shortened the delivery cycles of projects across numerous organisations with his expertise in taking complex technology problems and simplifying them to their essence. Andy's expertise in this area has enabled 5 fold reductions in cycle times on large scale projects. With over 15 years of experience, including significant periods as a team and management coach, a developer and a system administrator, Andy has been bridging the communication gap since before DevOps had a name.

Jan Molak is a full-stack developer and coach who spent last 12 years building and shipping software ranging from award-winning AAA video games and MMO RPGs through websites and webapps to search engines, complex event processing and financial systems. Jan's main focus is on helping organisations deliver valuable, high-quality software frequently and reliably through implementing effective engineering practices. A prolific contributor to the open-source community, Jan is the author of the Jenkins Build Monitor helping thousands of companies worldwide keep their builds green and the delivery process smooth.

Rate this Article

Relevance
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Is this worth the effort? by brian dsouza

Hi All,

Think its a noble initiative to build out this test approach but is it really worth the effort? Specifications expressed by the user of a system are the ultimate means to understand if a system works for the user. We then need a quick simple means to convert these into an executable format. The act of conversion must be simple and fast and even (in at least 50 -70% of the scenarios) done by the users themselves with little or no knowledge of the test target. In other words those complexities must be abstracted from the tester.
There are a bunch of tools available that will do exactly that. Write in Gherkin and convert to executable format with no knowledge of the complexities of the test targets as well as the ability to test across disparate test targets ( mobile, web, API) within the same test ( the ultimate user test) .
In all of this, there is no developer or code or framework to be built and the promise of in sprint testing is absolutely achievable.
Using the approach I mentioned above, these tests are written before the developer finishes coding and the developer cannot check in their code unless the tests pass. And in all of this, the developer had zero work to instruments tests to satisfy the gherkin statements.
Why would we be asking people to invest time and money to build out a framework for testing when we have existing tools available that can be used by non-programmers?

Thanks
Brian Dsouza

Re: Is this worth the effort? by Fadzlan Yahya

I'll try to answer. I think the promise of page objects or any of the improvements basically are acknowledging that functional test are much harder to maintain than to write one.

Re: Is this worth the effort? by brian dsouza

I could not agree with you more. Its the trap all automation testers fall into. But what is required is that you absolutely apply software design principles to your tests irrespective of whether they are written in Java or as a script. Separation of concern, cohesion, least coupling are critical. Test organization is also important and must match the product feature set. Your test design must ensure that you write once and reuse many times. Test data must be abstracted from the test so that the test can be used in different data contexts without any change to the test itself. Tests must be able to be executed in various contexts (scenarios) and behave differently depending upon the same. So a validating script in one context can become a navigation script in another context ( write once, use multiple times).
All of this is absolutely achievable as long as a proper process is followed. Whether you are writing a test or a piece of code, the lack of coding standards and guidelines, or the failure to have a design step will potentially result in an unmanageable artifact.
Test design must follow the same rigor as good code design but it does not need programmers to instrument code.

Write maintainable artifacts and the cost(time) of maintenance will be low.

This is the holy grail as we move towards faster ways to deliver product. Having a product person express their need in a test allows for outcome testing well before the code is fully written and provides very early feedback on usability etc. Having developers create interfaces and mock objects early helps this process and this is time and money well spent as we are unearthing the true intentions of the users and are able to tailor our solution to truly meet their need.

My point is that for all of this, we do not need high-end coders to be involved in instrumenting test code. I have done this multiple times and proved that this approach(testers write tests and developers write code) is an excellent marriage of a development mindset and a QA mindset, which however much people may argue that they are not, are different.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

3 Discuss
General Feedback
Bugs
Advertising
Editorial
Marketing
InfoQ.com and all content copyright © 2006-2016 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT