Performance Engineering in an Agile Project
Performance Engineering is an important software development discipline that ensures that applications are architect-ed, designed, built and tested for performance. However, mostly in traditional projects, the scope of performance engineering is limited to performance testing where instead of identifying the workload patterns and improving the process leading to better performance, the focus is on beating the clock under tested conditions. Balasubramanian,P shares an interesting perspective on building performance engineering practice in Scrum.
Balasubramanian provided the following overview of performance engineering activties,
- Collecting and validating the non functional requirements
- Developing the required models for analysis
- Developing a performance test strategy
- Reviewing the architecture and application code to ensure compliance to performance best practices.
- Reviewing the deployment of the application, and
- For pre-existing applications, reviewing the design and code to suggest appropriate tuning activities
According to him, performance engineering should be an essential activity in Agile because
- It provides immediate feedback about the system in each sprint
- It is easy to develop a false impression about performance – This is particularly true in Agile projects where a lot of focus is on delivering business value in each iteration. This usually results in diminished focus on the system quality attributes.
- Refactoring may introduce code that performs poorly.
- The goal is shippable, production quality code - Performance engineering helps in ensuring that the system is designed, built and validated against the required 'Quality of Service' requirements.
Balasubramanian suggested focusing on the following four phases for introducing performance engineering in Scrum.
I. Planning Stage
Understanding requirements and planning for performance engineering activities.
- Use cases and Performance – Understanding the system qualities associated with every use case.
- Performance Test Strategy – Detailing out scope of performance testing, infrastructure needs, tools, licensing cost, metrics and results format.
- Identify Performance engineering activities in product backlog - Activities like workload modeling, performance testing, performance assessment etc should be a part of the baklog so that they are not missed.
II. System Architecture Phase
Validating the architecture in terms of system qualities apart from functional and business requirements.
- Architecture Assessment – The Performance viewpoint – Keeping a focus on quality, the architecture must be assessed using either of
- Architecture Tradeoff Analysis Method
- Cost Benefit Analysis Method
- Active Reviews for Intermediate Design
Building blocks for producing shippable, production quality software.
- Code right – Apart from regular coding guidelines there should be guidelines about achieving good performance.
- Create Performance Unit tests - Developers should write unit tests that tests the performance of the application developed in a sprint. Though, initially the benefit would be minimal because the system is evolving. However, as the maturity of the system grows the performance unit tests become more useful.
- Review Design and Code – Automated code reviews using tools like JProbe, FxCop to test for performance issues related to code.
- Identify and Automate Performance Tests – Performance test should be identified and automated if possible for the 'Sprint n' in 'Sprint n-1'.
- Identify bottlenecks and fix performance issues – Performance issues identified in 'Sprint n' should be fixed in 'Sprint n+1'.
- Performance Engineering Sprint – Finally, the teams should consider having a parallel performance engineering sprint for every 2-3 normal sprints. Activities like performance testing, performance assessment, capacity projection can be carried out in this sprint
IV. Closure Phase
Deploying the complete application in the production environment.
- Setup Performance Monitoring Systems – The production system should be monitored using tools like JAMon to report performance issues in real time.
In order to make the adoption process easier, Scott Barber provided a detailed collection of tips to the performance specialist on how to engage and be productive in an Agile project.
Balasubramanian does acknowledge that there are multiple challenges in building a performance engineering culture in any project. The biggest challenge being the mindset change to focus on system quality attributes just like the system functionality. However, the rewards for building performance engineering practice right from the start far outweigh the investment and the benefits start multiplying with every sprint.
Agile no more vulnerable
All the best analysis, design, and coding practices simply won't help you with some of the hardest performance problems. The hardest performance problems often arise out of something that's being done that is subtly different this time around than the last 200 times you did it. What worked in the past isn't working in this situation, the only way to reveal that is with the code.
The good news is that if you are planning on frequent performance checks, those iterative time boxes actually work in your favor. You are testing and using the system a lot more and so you have more opportunities to see problems early, and therefore fix them before people forget about the code in question. Granted, that won't always help, but it will in at least some cases.
Re: Agile no more vulnerable
No Performance Specifications?
"This is an area everyone would agree is important and critical, but is the one often missed or partly
done due to various constraints. Efforts spent here will help the team to validate the performance requirements and be realistic about it."
Really? Important but often partly done? Sort of like the coverage this gets in the article!
It's a shame the article skips right over what's arguably the most important part of performance testing, clearly defined expectations of performance, and instead give it two sentences about being "important" while not providing examples, best practices, tips on how to clearly define performance levels, resources levels, etc.
Perhaps adding something like Planguage for defining Non-Functional Requirements would be an improvement to this method? More details here:
Ok with the activities, not ok with the methodology
Fernando Poblete Arrau
The product owner often don't have the knowledge to prioritize performance engineering items.
Ian Culling, Andy Powell & Lee Cunningham Dec 11, 2013