Test automation is often seen as a way to reduce the costs of testing, increase test coverage and effectiveness, and shorten testing cycles. In fact many software organizations consider automation as a vital step in establishing a mature QA program and it certainly has a lot of value if it can be effectively leveraged. However, it can never eliminate the manual testing, which is crucial for thorough testing of software applications, completely.
Automated testing involves higher upfront costs and should be looked at as a long-term investment where the pay-offs come anywhere between 2-4 years down the road. One has to keep in mind that there are various intangible benefits associated with automation. Performing a return on investment (ROI) for your planned automation can however help you understand right at the beginning the actual returns that you will get from your investments and you can weigh those against the benefits you will gain from automation.
To that end, Aspire Systems has developed a Test Automation ROI Calculator with which you can determine a high level approximation of effort and cost. The test automation calculator comes up with an approximation of effort and savings if you choose to move towards test automation based on the following factors:
Product Details
-
Type of product
-
Age of your product
Technology
-
Presentation layer
-
Middleware
-
Database
QA Team Size
-
Manual testing team size
-
Automation team size (if any)
Release Details
-
Number of releases planned for the year
Test Execution Parameters
-
Number of existing regression test cases
-
Number of configurations to be tested
-
Required number of regression test cycles per release
Cost Details
-
Hourly cost per QA resource
-
QA environment cost
Test Automation Tool
-
Test automation tool cost
Test Automation Parameters
-
% of test cases that can be considered for test automation
-
Estimated base time required to build test automation suite
-
Less: Usage of reusable components
-
Add: Time required to design the automation framework
-
Add: Time required to build batch scripts
Cost Details
-
Hourly cost per test automation resource
Automation is a strategic decision and an understanding on the potential ROI will help in the decision to invest in automation. This tool may be help you determine what works for you.
Community comments
Ha ha very funny
by Aslak Hellesøy,
Re: Ha ha very funny
by Amr Elssamadisy,
Re: Ha ha very funny
by Narayanan Moorthy,
Estimated time/cost for maintenance? Estimated gain from reduced cyce time?
by Eric Schumacher,
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Sai Venkatakrishnan,
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Amr Elssamadisy,
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Narayanan Moorthy,
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Narayanan Moorthy,
100% automation coverage
by Michel Löhr,
Ha ha very funny
by Aslak Hellesøy,
Your message is awaiting moderation. Thank you for participating in the discussion.
But April fools was 2 days ago...
I filled out the form, and was unable to enter anything in field 5 from bottom. Hit calculate and got no result, no error message and all my data was gone. If this thing was tested, it was done poorly.
Nothing to see here. Move on.
Re: Ha ha very funny
by Amr Elssamadisy,
Your message is awaiting moderation. Thank you for participating in the discussion.
:) Try something other than firefox. Good call. It works with Safari on my mac but no with firefox.
Estimated time/cost for maintenance? Estimated gain from reduced cyce time?
by Eric Schumacher,
Your message is awaiting moderation. Thank you for participating in the discussion.
I might have missed it, but where is the number for estimated time for maintenance and updates? The long term care and feeding is costly as well.
Where are the bits that estimate the dollar gain from reduced cycle time relative to shortening delivery/increase in quality?
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Sai Venkatakrishnan,
Your message is awaiting moderation. Thank you for participating in the discussion.
I am not able to believe this. On one side infoQ provides news about Lisa Crispin's Agile Testing book and on the other side we have something like this which totally contradicts that view.
I have tried all this ROI stuff and even was involved in developing one before and there are several things which are clear from them.
1) They make an assumption that testing can be done after development. Ir encourages test last attitude
2) You can't do automation without manual regression. Automation is about automating manual test cases.
3) I never advocate them as I don't think things are so deterministic.
These stunts are used by offshore test vendors to attract clients (I had worked with such a vendor before so this comes out of experience). Please read this beautiful post by Elisabeth Hendrickson about how Agile changes the equations in Automation. testobsessed.com/2009/02/10/how-much-to-automat....
I hope infoQ helps people understand ways to build better software rather than post every news which comes to its way.
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Amr Elssamadisy,
Your message is awaiting moderation. Thank you for participating in the discussion.
First of all, InfoQ does not recommend this tool, nor does it recommend against it. It is news - information that we thought our readers would find useful.
Secondly, from this newswriter's point of view, there is no black or white, everything depends on context. If you have a problem with the tool, by all means, don't use it. On the other hand, if you are considering recommending automating a large percentage of manual tests on your current legacy project, it may be useful.
As for me, I found the tool intriguing and haven't really spent enough time on it to understand the assumptions it makes. Nonetheless, automating existing tests is a recommendation I make to most teams I work with (not all, because that depends on context). Talking to executives in their language, ROI, is always a plus.
Finally, automating existing tests written by a QA department is independent of TDD. This assumes a QA department, which, by the way, many teams have and even when they go to cross-functional-teams, will still have testers.
100% automation coverage
by Michel Löhr,
Your message is awaiting moderation. Thank you for participating in the discussion.
I my current project I am responsible for QA (also process). And I have created a small testframework based on Groovy which uses a (german) DSL to test the Java Swing based application. Before my arrival there were no automated tests but now every developer writes their own GUI(unit)test which we put in our regressiontest in Hudson. This DSL allows for a testcase to almost look alike a testcase design for a manual testcase which is very productive to use.
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Narayanan Moorthy,
Your message is awaiting moderation. Thank you for participating in the discussion.
Re: Estimated time/cost for maintenance? Estimated gain from reduced cyce t
by Narayanan Moorthy,
Your message is awaiting moderation. Thank you for participating in the discussion.
I am not sure how you are just saying this ROI calculator is a stunt. This might be developed to for the purpose of serving the testing community. I think it tries to bring more awareness and i dont think there are ROI calculator available on the web similar to this. If you could generate the report from their site, u can see the value. Try once again.
Re: Ha ha very funny
by Narayanan Moorthy,
Your message is awaiting moderation. Thank you for participating in the discussion.
Now this works in all the browsers viz., firefox, IE, Safari and Chrome. You can use this ROI calc to find the ROI.