Design For Hybrid Agile Adoption

Posted by Upadrista Venkatesh on Sep 01, 2011

Sections

Process & Practices

Topics

Adopting Agile ,

Process Adoption ,

Offshoring ,

Distributed Teams ,

Agile in the Enterprise ,

Business Models ,

Adoption ,

Teamwork ,

Agile ,

Diversity in Teams ,

Business

Offshore Development is a critical success factor for many organizations as well as adopting Agile methodologies. However, these two techniques have never worked well together. The answer to this challenge - "Design for Hybrid Agile Adoption (DH2A) methodology" - is the focus of this article.

Introduction

Traditional software lifecycle models, such as waterfall, have declined in popularity in recent years due to the increasing demands on projects to accommodate changing business requirements, to control ever-larger and more complex requirements, and to reliably deliver the expected business results. The rapidly changing business climate is placing new and even more stringent constraints on projects with respect to time, cost and quality.

In order to deal with these recent changes new software lifecycle models have emerged that promise to keep pace with the industry. Agile is one such emerging methodology that is providing: better project transparency, better requirement trade-offs, faster time to market, and improved quality and reduced defects. While Agile approaches have clearly seen judicious success in recent years, these have primarily been associated with collocated teams. Unfortunately, collocation is not an option for all Agile teams. In fact, collocation is an approach, which, in some cases, has moved the industry paradigm decades back defeating the overall advantages and cost controls of distributed development.

Overcoming these challenges, "Design for Hybrid Agile Adoption (DH2A)", is a methodology defined to successfully execute Agile projects in a distributed and out-sourced environment.

This article provides an overview on the Design for Hybrid Agile Adoption (DH2A) methodology and discusses the following key practice areas:

• Overcoming the challenges of a fixed price model
• Minimizing the impact of "cultural difference" between distributed teams
• Improving the coordination and collaboration between distributed teams
• Measuring the success of your DH2A project
• Rolling-out DH2A across an organization - (i.e. Enterprise DH2A - e-DH2A)

The DH2A Methodology

The DH2A derives its life cycle approach based on several successful experiences executing Agile projects in an onshore-offshore and outsourced environment. These successes have helped define a new paradigm in Software Engineering called Distributed Agile. While several mature models exist today for onshore-offshore and outsourced projects, most of them combine non-agile approaches with the off-shoring strategies while others combine Agile with collocated strategies. DH2A is unique in that it marries the worlds of offshoring, out-sourcing, and Agile and specifically addresses the challenges of Distributed Agile.

The DH2A methodology defines a four segment approach (Appraisal, Estimation, Planning, and Implementation) as illustrated in Figure 1.1 and draws its success from the different patterns, best practices, processes and procedures designed to help distributed teams succeed with Agile.

(Click on the image to enlarge it.)

Figure 1.1: DH2A Methodology

Appraisal Segment

The first segment of the DH2A Methodology is the "Appraisal Segment". There are two phases in the Appraisal Segment named "Value Analysis phase" and "Project and People Assessment phase". The Value Analysis phase is used to understand the benefits an organization can achieve by adopting the DH2A Methodology. After the benefits are justified, the next phase in the Appraisal Segment, the Project and People Assessment phase, commences. This phase assess the AS-IS project and people characteristics against the DH2A requirements. The output of this phase is a recommendation on whether or not to adopt the DH2A Methodology.

1. Value Analysis phase: This phase determines if DH2A will provide value to this project. In this area, the project characteristics are assessed to determine whether the project has requirements to achieve faster Time to Market, whether the project has a high degree of requirements variability and unpredictability which needs to be overcome, if the project needs higher visibility, etc. If the answer to these questions are ‘YES’ then DH2A recommends adopting the methodology as it provides superior results in these areas.
2. Project and People Assessment phase: This phase determines if a project team agrees to meet the demands of DH2A. The demands that DH2A places on a project include:

• Business is available to teams at all times
• The project teams should adhere to the Team Composition Principle which mandates only DH2A experienced resources should be part of a DH2A project
• The team must adopt Test Driven Development, Refactoring, Continuous Integration and Pair Programming

If projects agree to these demands then DH2A is the right fit for these projects and should achieve superior results with DH2A.

Estimation Segment

The second segment of the DH2A Methodology is the "Estimation Segment" which defines an approach to estimation called "DH2A Spry Estimation". DH2A Spry Estimation is an approach to executing DH2A projects in the fixed price model, which is a unique feature of the DH2A methodology. DH2A believes that most projects can be executed successfully with fixed cost, time and scope constraints, if planned diligently.

A combination of the "Appraisal Segment" and the "Estimation Segment" facilitates the "Bidding/RFP Process" and customers have the luxury to select their vendors not only based on their experience in DH2A Methodology but also based.

Many organizations today choose to live with the limitations created by the Agile philosophy of leaving the project scope (or time and cost) open ended. The DH2A Spry Estimation Methodology is used to estimate fixed costs on a project, as well as determine the scope and timeframe of the project.

An organization should not begin a project without having minimal clarity regarding what it wants to achieve out of the project. In order to start, a project will need to have a minimal level of clear requirements - whether or not the project is executed using Agile methodologies or traditional approaches. In addition, a point to note is that most of the projects executed in traditional approaches have also been successful, and all of these projects are estimated up front using different estimation techniques, such as Functional Point estimate, Use Case estimates, or the Work Breakdown methodology. The major challenge with traditional methodologies is their inability to handle changing requirements.

DH2A Spry Estimation Methodology is based on a similar fundamental concept that any project will have at least minimal clarity on the basic set of requirements. However, for each unique requirement, the level of clarity may differ. The concept of the Spry Estimation Methodology is defined based on the clarity level of each requirement in a project. For example, consider Requirement A, which is partially clear to the teams after the initial brainstorming with the business users. Based on past experience and the level of clarity of the requirement, the team estimated that Requirement A would consume around 20 hours of effort. As per the DH2A Spry Estimation Methodology guidelines, based on the level of clarity for Requirement A, the team and the project stakeholders agree to a deviation of +25% and -10% on the estimated effort. This means that during development, if the team discovers that Requirement A will take more time than estimated, they can safely consume another 5 Hours of effort for Requirement A and still be within the quality norms of the DH2A Methodology. Based on the level of clarity on the requirements there is an acceptable deviation (in effort and cost) agreed between the stakeholders and the project teams based on which the final estimate and cost is derived.

Planning Segment

The third segment of the DH2A Methodology is the "Planning Segment". Planning before development begins has always been considered critical project success and is the core theme of the "Planning Segment" in DH2A. This segment of the DH2A methodology provides an opportunity for projects to:

• Create the right team structure:
  DH2A provides a "Team Maturity Index" tool to help define the optimal team composition for this project. It uses the different characteristics of the project as inputs and defines a roadmap to what the project team composition should be between onshore and offshore. There are two parameters based on which the team composition between the distributed locations are determined. The first one is the team’s knowledge in Agile. Another important aspect is understanding the functional and technical competency of the development teams in the underlying project environment, which in DH2A we refer to as Operational Knowledge. A combination of both of these aspects determines the team mix between distributed teams. DH2A has defined a simple tool called the DH2A Stage Verification Tool, which should be used by projects to determine their current ‘Resourcing Stage’. This tool maps the capabilities of a project in its DH2A Methodology maturity along with the operational knowledge of the teams to arrive at the Resourcing stage that must be adopted by a project.
• On-board past experiences:
  The project team’s best practices and lessons learned are reviewed and integrated into the current project proposal.
• Create a plan to adopt the Engineering Practices:
  Engineering practices such as Continuous Integration, Pair-programming, Refactoring and Test First Approach are critical to the success of any project. DH2A provides the "Engineering Practices Maturity Index" tool to help identify the key Engineering Practices that should be emphasized on this project. It uses the different characteristics of the project as inputs and defines a roadmap to adopting the DH2A Engineering practices. There are four levels of maturity for the engineering practices defined by the DH2A Methodology (Level 0 through Level 3 which are described below and illustrated in Table 1).

TECHNICAL PRACTICES	LEVEL 0	LEVEL 1	LEVEL 2	LEVEL 3
TEST FIRST APPROACH	Not Followed	Performed only for some critical scenarios. Testing may not be automated yet.	Performed for all scenarios. Testing may not be automated yet.	Performed for all scenarios. Test Automation and execution performed on all scenarios.
REFACTORING	Not Followed	Done at the end of the iteration. May not be automated yet.	Done at the end of every scenario development. May not be automated yet.	Done at the end of the day. Automation is mandatory.
CONTINUOUS INTEGRATION	Not Followed	Components are integrated after completion using manual integration on a daily / weekly basis.	Integration performed daily with the automated integration tests.	Integration is done with automated builds, automated integration tests, and automated deployment.
PAIR PROGRAMMING	Not Followed	Performed by two developers for some critical scenarios.	Performed by two collocated developers for all critical scenarios. Remote Pair Programming not yet planned.	Performed by two collocated developers for all critical scenarios. Remote Pair Programming accomplished for all scenarios meeting the DH2A Remote Pair Programming standards.

Table 1: Engineering Practices levels

Level 0: Indicates that:

• None of the engineering practices (i.e., Test Driven Development, Refactoring, Continuous Integration, and Pair Programming) are being followed. A project assessed at Level 0 should advance to Level 1 in the current DH2A project within 60 days after the project is initiated. A plan should be defined to adopt subsequent levels.

Level 1: Indicates that:

• Test Driven Development is being performed only for some critical scenarios and the testing process is not automated.
• Manual Refactoring is performed at the end of an iteration.
• Integration is being performed on a weekly or daily basis, after a few scenarios are developed without using any automated tools.
• Pair Programming is performed very rarely by two collocated developers for some critical scenarios.
• A project assessed at Level 1 should progress to Level 2 in the current DH2A project within 60 days after the project is initiated. A plan should be defined to adopt subsequent levels.

Level 2: Indicates that:

• Test Driven Development is being performed for all scenarios and the testing process is not automated.
• Manual Refactoring is being performed after coding is completed for every scenario.
• Integration is being performed frequently (daily) for every scenario without using any automated tools.
• Pair Programming is being performed by two collocated developers for all critical scenarios. Remote Pair Programming has never been planned.
• A project assessed at Level 2 should adopt Level 3 within 60 days after the project is initiated. A plan should be defined to adopt subsequent levels.

Level 3: Indicates that:

• Test Driven Development is being performed for all scenarios and testing is automated.
• After coding is completed, Refactoring is being performed for every scenario using automated tools.
• Continuous Integration is being done on a daily basis using automated tools.
• Pair Programming and Remote Pair Programming are being performed for all critical scenarios, as per the Remote Pair Programming guidelines, by distributed teams using DH2A recommended tools.
• A project assessed at Level 3 should continue at this level and demonstrate improvement.

DH2A provides a simple tool called the "DH2A Engineering Practices Maturity Verification Tool" which accepts as input the team’s maturity and level of experience in the four engineering practices of Test First Approach, Refactoring, Continuous Integration, and Pair Programming. The tool then determines the current level of maturity of the teams and recommends a maturity level to be adopted for the project.

Create a plan to address the Communication needs of the project

To be successful, the project team should have a variety of communication paths established to ensure constant and successful communications are possible between all team members. These communication paths should include options such as: dedicated phone lines, video conference facilities (at both onshore and offshore locations), instant messaging, etc.

(Click on the image to enlarge it.)

Figure 1.2: Video Conference set-up for real time collaboration

Real-time collaboration through video conferencing, as shown in Figure 1.2, will greatly boost the team’s productivity and help minimize the cultural differences between the onshore and offshore locations. To be successful, the video conferencing setup must be located in the development team areas and be connected at all times between the onshore and offshore locations such that face-to-face collaboration can occur naturally and frequently throughout the day. The onshore and offshore members of the team should be able to see each other just as they can see their collocated team members. This is possible for teams, which have an overlap in time zone between the distributed locations. For cases where an overlap in time zone is not possible, DH2A recommends that the teams stretch their time and conduct the meetings from home over webcams.

Define Critical Success Factors

Define a plan for using the various metric management procedures defined by DH2A to measure the success of your project. One of the toughest things to do well in today’s rapidly changing and dynamic software development environment is to effectively measure and assess the performance of the project. DH2A assesses the success of a project based on parameters such as Schedule, Cost and Time measured with the baseline data for the project. As an example, the Key Process Areas (KPA) of the Cost Management function mandates that every DH2A project analyze past data and arrive at a baseline of cost deviations in previously executed projects. These can be from a combination of several projects executed in any methodology. This baseline cost deviation is termed the Elapsed Cost Deviation, in DH2A terms. The Elapsed Cost Deviation forms the baseline for the DH2A measurement of success for the Cost Management KPA.

The Cost Management KPA also requires that cost deviations are calculated and managed for every iteration during the course of project execution (that is, in the Implementation Segment). Cumulative Cost Deviation is defined as the total Cost Deviation for all iterations completed to date. The Cumulative Cost Deviation should be compared to the Elapsed Cost Deviation, which demonstrates the results achieved with the DH2A Methodology when compared with other methodologies.

• The DH2A approach for metrics management is always a comparison of the past data with the actual results.

Implementation Segment

The last segment of the DH2A Methodology is the "Implementation Segment". This final segment ensures that all project features are developed and deployed into production per the DH2A guidelines and are built on the core practices to avoid waste in software development, which directly improves the project success rate. With several practices such as Upfront Architecture (DH2A demands that a full blown architecture is developed before the project starts without doing it in an iterative way), adopting completely the engineering practices, etc. the DH2A methodology clearly differentiates itself from other Agile Methodologies.

The Enterprise DH2A Methodology (e-DH2A Framework)

The success of a project is often dependent on the people, processes and capabilities that cut across multiple disciplines. To consistently realize project success, proven best practices in these areas should be implemented across the enterprise. However the leap from a small success to organization wide success has never been an easy step. Several experiences in several lines of business have demonstrated that the big-bang implementation often leads to failures. While some exceptions exists, the reasons for these failures are due to the lack of proper and consistent integration among defined disciplines, processes, and capabilities which results in sub-optimization, confusion, and potentially unnecessary expenditure.

(Click on the image to enlarge it.)

The Enterprise DH2A Framework (e-DH2A Framework) defines an approach to adopting DH2A across an organization. The success of e-DH2A Framework is derived from two critical activities:

1. Define an organization level Project Management Office (PMO) that monitors and manages the DH2A projects - (referred to as the e-DH2A Office). For organizations, which already have a PMO, DH2A has clear steps for tailoring the PMO such that DH2A projects can be monitored by the PMO.
2. Follow a three-staged approach to rolling-out DH2A from one project to multiple projects. The following stages allow the DH2A Methodology to be realized across an enterprise:

• Simple Stage: A single project adopts the DH2A Methodology in line with DH2A guidelines.
• Compound Stage: Multiple projects adopt the DH2A Methodology.
• Absolute Stage: All projects that meet the DH2A requirements implement the DH2A Methodology.

The core fundamentals of e-DH2A revolve around setting up the e-DH2A Office in alignment with the existing PMO. Transforming an organization to the e-DH2A should yield the following benefits:

• Higher customer and team satisfaction
• Lowering cost
• Higher productivity
• Increased Qualitative benefits
• Increased Quantitative benefits
• Faster release cycles

Conclusion

In conclusion, the DH2A journey requires lot of rigor and passion to fully realize the benefits to your software development teams. While there is clearly an initial investment to implementing the DH2A (since it demands both a cultural and structural change to your organization), the benefits realized in the long run are well worth the investment. Like any other business transformation, successful implementation of DH2A requires a steady, ongoing, stepwise approach. If you choose to implement DH2A, in time your organization will be able to reap the full benefits of Agile while also taking advantage of the proven cost controls of offshoring and outsourcing.

About the Author

Upadrista Venkatesh has worked in a blended mixture of the onsite-offshore model for many years and has been the trusted advisor for several organizations - helping them adopt the strategies of distributed development. He is currently working in a leadership role for a large IT service provider and is passionate about, and a respected expert in the distributed development arena. He has been a guest speaker to Harvard Business School (speaking on the topics of general management) and speaks in multiple conferences especially on the success of distributed development. He has guided several customers towards successful Agile implementation in distributed environments and is author of several books including: "Managing Offshore Development Projects - An Agile Way".

Upadrista can be reached on his email at: venkatesh@venkateshupadrista.com. Please also visit this link to download the DH2A tools.

The DH2A Methodology has been released into the market via the book "Distributed Agile: DH2A - The Proven Agile Software Development Approach and Toolkit for Geographically Dispersed Teams" and is available here.

1. Back to top

   agile?

   by Neil Murphy

   What is agile about this methodology? "DH2A believes that most projects can be executed successfully with fixed cost, time and scope constraints, if planned diligently." Almost by definition this is a non-agile approach. Most of the things mentioned are about rollout of the method or about imposing controls external to the project team (e.g. the PMO),
   
   Thoughtworks are the thought leaders in distributed agile and I have read of other companies making it work. Not impressed by what is written above and the price of the book is exorbitant. Drop the price by 50% and make it available as an elecetronic download and I might look.

   Reply

2. Back to top

   Re: agile?

   by ShriKant Vashishtha

   It looks to me like yet another complex process in doing distributed Agile. It makes very complex to think team in terms of levels. Why not introduce things from the very beginning. I don't see any mention of distributed pair programming over here which is essential in having "One Team" feeling across geographies. Article has been written more from project management point of view which may be helpful for PMs coming from Waterfall background. However important stakeholder i.e. team is not involved in terms of the implementation. As mentioned in above comment there are already successful implementation of distributed Agile (Xebia for instance - citeseerx.ist.psu.edu/viewdoc/download?doi=10.1...) where the focus is to just fix the problems introduced by distributed nature of team instead of redefining yet another process from scratch. So you have all Scrum meetings but on Skypy/google+ hangout. Apart from local pair-programming you have distributed pair programming. And focus is a bit more on distributed peer communication (xebee.xebia.in/2009/12/29/the-nutbolt-pattern-f...) as team members don't see each other face to face. So these are some additional practices to avoid problems posed by distributed mode. But otherwise the fundamental principles of Scrum and XP are same.
   
   twitter.com/vashishthask
   www.svashishtha.com

   Reply

3. Back to top

   A very impressive method - DH2A

   by paul Hamilton

   Initially like others, after having a glance at this article, i assumed DH2A as a heavy process oriented methodology and a mix of SCRUM plus breaking the agile manifesto . But since my programs execute in distributed agile, i thought of giving a try reading this book in detail. I purchased the book few weeks back and completed reading it today.
   
   WOW ! MY PERCEPTION HAS COMPLETELY CHANGED NOW............
   DH2A is a very niche methodology and with the kind of best practices very nicely orchestrated to bring together in the methodology, DH2A is definitely a methodology to be followed if we want to make agile successful in a distributed environment. Until today, our industry lack such a methodology and i sincerely wish this to be a big success and purely because it carries a lot of worth.
   
   BTW, i am not a fan of the author nor an investor in his company :) ... & hence I have given a unbiased personal view.

   Reply

• Older >