Second Functional Test Workshop Results
The second Agile Alliance Functional Test Workshop was held as a pre-conference session before Agile 2008. It was run as a series of open space sessions facilitated by Jeff Paton. The primary purpose of this workshop was to discuss cutting-edge advancements in and envision possibilities for the future of automated functional testing tools.
In response to the purpose the group created a diverse list of open space sessions:
- Narrative Testing – what is and how can it help.
- Why hasn’t Acceptance first development taken off.
- Tests vs. Specifications – what is we’re writing?
- Tool Clearing house? Why so many tools?
- User Understandable Tests vs. the Power of Programming
- HTML: ideal testing DSL or an abomination?
After a lunch the group conducted a “Futurespective” – a retrospective looking a year into the future. The goal was to identify things that we wanted to happen in the next year. The group discovered several big needs, among them articles that explain current best practices in functional testing and the distinction between test frameworks (tools that run and report tests) and drivers (tools responsible for translating the tests into the language of the system under test).
On the subject “Why hasn’t Acceptance Test Driven Development Taken Off”:
- miss the fun / aha! moments / benefits that they get
from UTDD (Unit Test Driven Development).
- there is a difference in scope and cadence of tests.
("Cadence" became a key word people kept coming back to.)
- laborious writing of fixture, which doesn't feel as valuable as "real
- don't see the value (or ROI) from ATDD (Acceptance Test Driven Development)
- they are not used to working at that level of precision
- no time
- they prefer rules to examples
The session on Tools resulted in agreement to build a clearing house for Agile Functional Test Tools. Among many attributes the group decided to characterize tools by certain attributes:
- Test Input Format (XML, HTML, Java, Groovy, Vendor Script)
- Support for Writing a Domain Specific Language
- Test Target: Windows Application, Browser based, WinForms, Eclipse Rich Client
Work is just beginning on classifying the tools.
Additional session notes: Tests vs. Specifications/Requirements, Tests vs. Examples, Narrative Testing and pictures are available. Finally Mike Debbo has published his as AA-FTT 2008 workshop redux, part 1 and part 2.
Level of Precision
I had something of a Eureka moment when I read that. I've noticed that everyone but programmers and testers get annoyed when asked to focus on low-level details (including not just customers, but managers and even requirements analysts). I think this "impedence mismatch" of expectations between business and technical people is the primary cause of "scope-creep" - and we all know what that leads to. Managing expectations is difficult at best, and I have yet to see a business person get excited about co-authoring acceptance tests. Typically, management will attempt to separate techies and business folks when they notice any annoyance on the customer's part, so co-authoring falls to the wayside.
I'm interested to hear more about this, particulary discussions regarding business people deriving more benefit from acceptance testing.
Re: Level of Precision
I hope you're able to join us on aa-ftt mailing list (yahoo groups).