The Top Five Challenges of Building Software Platforms in the Agile World
During the past decade, agile software development has gained great momentum and found its way to an overwhelming number of organizations of different scales . Agile methods preach a raft of values and provide a wide range of practices that help reach and maintain such values. Although the initial focus of agile methods has usually been centered on the efficiency of the team as a unit of operation, recently there has been a movement towards scaling agile methods up to the enterprise level . Nonetheless, at the enterprise level, new concerns arise that may require revisiting some of the values and practices of agile software development. One of the major concerns is the issue of building software platforms as a strategy to achieve reuse at the enterprise level. In this article, I list the top five challenges that an agile organization should expect to face when deciding to adopt a software platform strategy.
What Is A Software Platform?
A software platform is a set of subsystems and interfaces that form a common infrastructure on top of which a set of related products are developed . Software platforms resemble a strategic way to achieve reuse at an organizational level. Many advantages of software reuse have been reported by organizations such as: fast delivery of products because less development and testing is required, reduced development and maintenance costs, improved quality of reused artifacts, reduced risks by reusing a previously proved solution, and better project estimates for time and cost.
The Top Five Challenges
Adopting a software platform strategy is a radical change that requires the organization to revise and adjust its existing practices. In the agile context, certain agile principles and practices create challenges if they are used without any reconsideration. In this article, we focus on five main issues, namely: feature teams, team autonomy, business value, code ownership, and stability. While the challenges we list here may not be the only ones, they seem to be the most evident. Also, the severity of the impact of each of these challenges on software platforms may vary from one organization to another.
1. Feature Teams Versus Component Teams:
Feature teams usually assume end-to-end responsibilities in a given system by orienting their work around features (aka. stories). On the other hand, component teams focus more on delivering a sub-system that interacts with other sub-systems in order to be useful. Due to the focus on delivering tangible value to the end-customer, agile methods favor feature teams over component teams but they do not necessarily deny the need for having component teams. However, a common perception amongst practitioners is that component teams are always disadvantageous. It is not uncommon to hear the following statement from some project managers or technical leads:
“It has been told to me that it is bad to have component teams in the agile world which are not end-to-end responsible.”
However, in the context of platform development, there seems to be a need for a combination of both. For example, certain services – especially backend services – are so fundamental and expensive that they may require a dedicated component team as opposed to being maintained by a feature team as part of an ongoing project.
A hybrid of feature teams and component teams working together is important. Some component teams are needed to build and maintain certain core services that affect a wide range of products or/and require specific domain knowledge or expertise. Other component teams that have a closer interaction with the application layer may need to collaborate more closely with feature teams in order to understand their requirements.
Feature teams usually assume end-to-end responsibilities to deliver application level features. To do so, feature teams will use services provided in the platform and on occasion may need to modify certain aspects to satisfy their requirements. Modifications such as these can take two forms:
A. Direct modifications to the core platform code:
Feature teams should be able to fill in the missing parts in the platform and possibly add new parts when needed. It is recommended though that this be done under two conditions:
- All changes by feature teams need to be audited by the platform team owning the part that needs to be changed or extended. This is to ensure that the change is technically sound and does not violate any design or architectural constraints or guidelines. Auditing should also ensure that the code is cross-platform when it needs to be.
- Feature teams should have access to the regression tests of the platform components so that they can build locally against different configurations before they commit their changes. Otherwise, the platform may become unstable and may be perceived by its users as unreliable.
B. Branching from the core platform code:
Certain change requests coming from the feature teams can sometimes create irresolvable conflicts with other products that use the platform. For example, a new product may require some behavior that is slightly different from the existing behavior of a certain component in the platform. If the organization finds value in supporting both behaviors, then the developers from the component and feature teams will need to collaboratively abstract the aspect of the component that is common to all products, and then exploit the aspects that vary in different products. Configuration management or variability management techniques  can be used to support this step.
If a certain requirement from the application level involves a change in the platform, it might be beneficial to have at least one member of the platform team (i.e. component team) join temporarily the feature team that is conducting the change. This member will validate the decisions made by the feature team before they actually take place. Such collaboration and validation will also make the auditing process faster. This participation will improve knowledge transfer in the organization because it is an opportunity for developers in the feature team to have a better understanding of the domain and learn about cross-platform issues. In a regular scrum meeting, each member in the platform team should report the outcome of their participation in the discussions with feature teams (e.g. decisions taken, required changes).
2. Team Autonomy:
Another issue that may hinder successful software platform development is high team autonomy. When members of a highly autonomous team stay together for a long time, there is a risk of this team gradually becoming a silo. Allowing silos to be created within an organization has serious consequences such as redundancy in code, insufficient visibility of reusable assets, false assumptions about the reused assets, and platform divergence. Platform divergence can occur when internal platform decisions result in different branches of that platform. After a while, the branches diverge so much causing the platform to become too specific to a certain operating system or product, or even causing the loss of a common underlying model.
Moreover, high autonomy has an impact on decision-making where the team considers certain issues internal without paying much attention to the consequences of their decisions on the underlying platform or on the feature teams trying to utilize their platform. When the teams and business units have a sense of federalism (a political system often seen in larger agile organizations), making decisions related to the platform becomes an even greater challenge. For example, sometimes a decision needs to be made on whether to reuse an existing platform or take a different direction such as building an independent variant to satisfy a certain business concern. On the one hand, the corporation that owns the software has economic reasons to push reuse, but at the corporate level it is often hard to see all the intricate details of the specific business concerns, hence making such decisions challenging. On the other hand, if these decisions are left to the teams and business units directly influenced by the reuse concern, a raft of issues may steer the decision making process in a wrong direction. For one, the individual teams tend to choose the direction where they see short-term gains as opposed to thinking about the long-term goals (e.g. sustaining the platform). There is also the “not-invented-here” syndrome that biases teams towards developing their own components as opposed to reusing others’.
3. Business Value Thinking:
In agile organizations, there is a strong emphasis on delivering business value. This has proven to be a great advantage of agile software development. Nevertheless, the immature yet common understanding of what constitutes business value can be a hurdle. This hurdle does not come from the business value thinking per se, but from the misconceptions that surround this type of thinking. For instance, the notion that ‘for something to have value it has to be visible to the customer’ is damaging, especially in the software platform context. It is not unusual to hear statements like:
“I would define value as useful and usable functionality delivered to the customer”
But business value is not always immediately visible or may not directly influence the customer. Take refactoring as an example.
The transition to a platform strategy may provide for many economical advantages for the organization, but from an end-customer’s perspective, nothing would change. In an environment where business value is defined strictly by the impact on the current customer, it is a challenge to motivate certain teams and individuals to buy into the platform. Teams who work on developing and continuously improving core services may be wrongly perceived as delivering less value than feature teams that have a firsthand interaction with their customers. Therefore, a more mature understanding of business-value is needed to encompass both customer-facing value and business-facing value. Instead of only asking the question: “How does this benefit the customer?” we should also ask the question: “How does this benefit the business?” The second question eventually stems from the first question in the sense that the success of any business relies on its abilities to satisfy their current customers and attract new ones. But, it is important to make that distinction explicit to correct inaccurate perceptions.
4. Code And Product Ownership Thinking:
Teams and product owners can be very protective of their assets and products making the transition to a platform strategy more difficult. This is mainly because teams owning a certain component would rather keep dealing with that component rather than retire it and maintain a shared component in the platform. Moreover, in the platform context, code ownership is not as explicit and clear as it is in silo software development. That is, it is not clear who owns a component in the platform that is shared across different teams and products.
One way to overcome this issue is to promote the idea of collective ownership of the platform and the products being built on top of it. This may require an internal open-source model that allows feature teams to contribute to the platform and assume temporary responsibility and ownership for the part of the code they contributed to until it is mature and stable enough to become the responsibility of a component team. Also, for product owners, a more collaborative approach is needed to decide how to relay requirements to their teams in such a way that considers the short-term wants of the customers as well as the long-term requirement of sustaining the platform.
5. Agility Versus Stability:
In traditional models of building platforms, a platform-then-product philosophy is dominant. This approach can be seen in paradigms like software product line engineering where there is an emphasis on developing domain artifacts and then application artifacts . This succession means that an organization does not start building products until the development of the platforms underlying these products has made considerable progress. Agile organizations, on the other hand, are more inclined towards adopting a product-driven approach to software platforms. In such an approach, platforms are derived from a number of existing products as well as from the requirements of ongoing projects that are considered the early adopters of the platform. In these projects, the products that rely on the platform are developed at the same time as the platform.
Especially in situations where the platform is product-driven, it is challenging to create a balance between the stability of the platform and the flexibility to accept change often. While, in an agile context, teams must remain responsive to the needs of the products at hand in order to satisfy the current customers, it is imperative to realize that platform stability is key, because many products rely on the platform as a common foundation. Therefore, it should be trusted and should not change very often. For infrastructure components and core services, being part of an ongoing product development effort with frequently changing requirements may impose great risks.
Another issue arises when a product owner requests a change in the platform that involves a cross-cutting concern such as usability. In this case, the challenge is to make a choice between two options. The first option is to follow agile principles and honor the change request to satisfy the customer at hand. In this case, all products relying on that aspect of the changing platform would be affected causing instability. The other option is to ignore the request until it proves to be an issue in other products too. But that may come at the expense of the satisfaction of the customer at hand.
Being able to determine the underlying business value of each requirement is the first step towards addressing the stability vs. agility dilemma. Weighing the business-facing value and the customer-facing value is essential to deciding whether to proceed with a request right away, delay it until it proves to be a common need, or even completely ignore it in favor of your business goals.
Scaling agile up to the enterprise level has consequences that we need to understand and deal with. Reuse by means of software platforms is one of the enterprise concerns that call for adapting some agile methods and practices to support the needs of the business strategy and goals. The five main challenges we face include: finding a hybrid team structure that supports component development as well as feature development, avoiding silo thinking and team federalism, revisiting the definition of business value to encompass aspects that do not necessarily face the customer directly, building a more collective and collaborative model for code and product ownership, and finally finding a balance between agility and stability based on the proper understanding of the business value of the requirements.
About the Author
Yaser Ghanam is finishing his PhD at the Department of Computer Science, University of Calgary. He also works as a sessional instructor teaching software engineering to undergraduate students. Yaser obtained his bachelor in Computer Engineering and a minor in Engineering Management from the American University of Sharjah, UAE. He has two concentration areas in Psychology and Applied Mathematics. Yaser has been granted a prestigious award for his doctoral studies, and received the 2009 NSERC Research Achievement Award. Yaser has been nominated for the Student Union Teaching Excellence Award in Winter 2010. In his PhD, Yaser is investigating how software reuse and customization can be achieved using agile methods. During the summer of 2010, Yaser was a visiting scientist at the University of Helsinki in Finland where he conducted field studies in leading IT and telecommunication companies. Yaser has published in a number of venues such as the Software Product Line Conference, the XP conference and the Agile conference. Yaser is also the guest co-editor for the upcoming special issue of the “Software: Practice and Experience” journal.
 Leffingwell, D. 2007. Scaling Software Agility: Best Practices for Large Enterprises. Addison-Wesley.
 McGrath, M. 1995. Product Strategy for High-Technology Companies. Homewood, IL: Irwin.
 Pohl, K., Böckle, G., and Linden, F. 2005. Software Product Line Engineering: Foundations, Principles and Techniques, Springer.
 van Gurp, J., Bosch, J., and Svahnberg, M. 2001. On the notion of variability in software product lines. Proceedings of the Working IEEE/IFIP Conference on Software Architecture, pp. 45-54.
There is even no discussion why component teams are bad for agile teams (actually for software development teams in general). If the author explored about the problems with components teams, the result would be more different.
5 Stability vs Agility
This is a false dichotomy.
Re: Too shallow
agile: component interface changes whenever that better fits a requirement
I'd say they are clear opposites.
Unless the requirements are stable. (hah!)
I really appreciate the thoughtfulness of this article.
5 Pillars of API ManagementCA Technologies
Mike Hartington Jul 26, 2015