Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Growing Agile… Not Scaling!

Growing Agile… Not Scaling!

Key takeaways

  • Why it is important to focus on growing your own agility instead of taking shortcuts and adopting someone else’s model
  • Which role culture plays in an agile transition and effectiveness of organizational change
  • How to make sure you are agile at heart, and move continuous improvement to an organizational level, to become a more agile organization
  • How to leverage your organizational structure to better focus on customer value and deliver what is important to them
  • How to increase your organization autonomy by decentralizing control and creating containers for empowerment

Growing Agile… not scaling!

I dislike the term scaling applied to agile, or better agility. The reason for my not liking it, is because it evokes metaphors which are bound to manufacturing such as: “scaling a plant”, “scaling production” and all the rest that can evoke assembling and building things.

I like to think about agility as something which goes far beyond building and assembling and also far beyond technology. I won’t spend too many words in arguing about the fact that Software Development has nothing to do with any “Production” related metaphors, but allow me to mention a couple of very common dysfunctions:

  1. Software development is a creative and problem solving activity, and unfortunately due to the fact that emerged out of hardware development, it inherited all sort of association to production, supply chain, waterfall and all the goodies about predictability, variations…
  2. Comparing software development to “production” caused the whole IT service industry to price development by the hour, like it were a mechanical repeatable activity, with linear impact on the outcome… this is particularly funny isn’t it?
  3. People consider writing code to be the “construction” part of creating software, while it is the “design” part instead. In fact the construction part, which is compiling high level abstraction languages into machine working code, has been mechanized and automated a long time ago, so we could easily say that software development has been fully automated since at least 50 years! But no… it doesn’t really stick, does it?

Now back to the point, given the aforementioned misunderstanding, our industry is plagued by multiple attempts of “scaling” agile, with a very strong - almost unique - focus on delivering software. We can ask a wide number of people familiar with agility, and I am pretty sure that we would get a wide spectrum of answers on the matter. I am confident though, that the majority of them would agree that what makes an agile team successful is not the “process” nor the “tools” but rather the way people developed an effective level of interaction with each other. Inspecting and adapting both the work done and the way it is done, seems to be the core of continuous improvement, combined with the usage of metrics. So if we were to focus only on “scaling” the practices, processes and tools and not on the mindset and culture, I am pretty sure we would not achieve long term success, sustainable pace and most of all, people satisfaction and engagement!

I like to use the term growing agility, rather than “scaling” because connects better with the fact that developing agility within an organization has more to do with an organic system, rather than with a mechanical one. If culture eats strategy for breakfast, then we have to recognize that the way towards agility, requires addressing culture and mindset as first class citizens. Over the past years, I have came to particularly appreciate the impact of culture on the effectiveness with which human systems operate. So growing agile, means both focusing on culture, and on co-evolution of practices and tools. In every high performing environment I had the pleasure to work, people were having control of values, principles, practices and tools.

Culture eats strategy for breakfast

While changing culture is a challenge for every organization, there are approaches and tools which can help understanding where one stands, and also designing together in which direction to evolve to create the right context to support the current strategy. Only in this way, changes will stick, and will be owned by the people within the organization instead of by the institution governing it. Already achieving an agreement on where the organizational culture stands, can provide incredible insights on solving issues, and surely strengthens the alignment.

For example with one of the companies I am working with at the moment, the leadership team took the cultural matter very seriously, and decided to make it explicit, because they have realized that it was the only possible way to make sure everyone would feel the same about moving forward. We started by “assessing” the current culture, which requires courage, and quite some structured conversations. We have used the Competing Value Framework (CVF) this time, but could have been any other model. Conversation between leadership team and other opinion makers, through the sharing of stories from the company past, and exchanging of opinions we came to the point of having agreed on a cultural profile. This is the first step, the second one is to understand based on the strategic direction the company is taking, what would be the best cultural context to support it. Also given that companies tend to identify themselves pretty strongly with one culture or another, it has been very important to identify also what to keep from the existing culture, that would provide value also for the future. This sends a very important message to the people, that isn’t “we are doing things wrong around here”, but rather “we are doing things great around here, and there are options to improve even more”. Some example stories have been chosen, which embody the value and the principles that the leadership agreed to be the right ones to support the next stage of evolution. Beside the stories, using the Cynefin Framework by Dave J. Snowden, we have managed to let emerge a series of Rituals (e.g.: Daily Scrum, Ritual Dissent, Planning Poker…) to strengthen the behavior the organization wanted more, and others to dampen the ones which have been considered inappropriate for the new direction. Through the combination of storytelling, and sharing new rituals, people had a concrete way to start shifting the culture, which impacted on their daily life too, adding more productivity as well as fun.

Continuous Improvement at the organization level

Effectively establishing continuous improvement to an organization at large requires to develop quite some discipline, infrastructure and most of all openness and trust. Working with agile42 allows me - and all of my fellows coaches - to spend a significant amount of time in analyzing data and cases from different case studies all around the world. Being directly present in 13 different countries, allows to collect diverse experience and information that we use to develop new frameworks and tools to support organizational change. We have been actively working on a set of methods to make continuous improvement sustainable and effective at an organizational level. For example, we have implemented an organizational continuous improvement and learning framework (It’s called the Enterprise Transition Framework – ETF, and it is available in cc 4.0 by-nc-sa license) that employs double loop learning, to coherently learn from empirical probes, and extends this learning throughout the organization. It starts by formulating hypotheses about how a certain change to the organization could benefit its performance, or improve people’s lives, or deliver more customer value. Once the hypothesis has been formulated, we need to look how it would fit into the current existing strategy, and also how it can be validated as fast as possible, at the lowest possible cost. To do so, we use another tool, the Agile Strategy Map (also available as cc 4.0 by-nc-sa) which identifies dependencies between different changes, initiatives, or strategic leverages. Once the connections between the new hypothesis and the existing strategies have been made, we facilitate a conversation with the leadership team to identify safe-to-fail experiments that would allow the organization to test these hypotheses, as fast as possible, and at the lowest possible cost. Once experiments are defined, we start looking for volunteers, as at this stage of uncertainty, you want to have people willing to succeed, and willing to face some impediments on the way. This motivates employees to participate in the definition of the improvements, as the only thing which is set is the goal, and some constraints.

The team will have to find out how to achieve the goal in a sustainable way. This way the organization learns about conditions that are necessary to succeed. We never suggest running only one experiment at a time, as that might yield false positives. Running multiple experiments in parallel, allows an organization to collect results on multiple options. It is a very good sign, if some of the experiments fail, it is actually a sign that the organization is really trying hard to change. Failing fast and failing often is an advantage. So how does this help an organization to improve continuously, and steadily? There is no trick, just discipline, and for this is necessary to establish a regular cadence at which what we call a transition team is spending time to evaluate ongoing experiments results, and linking those results back to the Agile Strategy Map. This allows the organization to learn from its own experimentation, and as a by product, helps to shape a continuous improvement culture. The organization learns to roll-out the results of successful experimentation, by evaluating the consequences of it, and the way it could alter existing equilibrium. For example, introducing a framework such as Scrum, requires organizations to establish new roles, which are normally not existing. So most organizations will fall into the trap of mapping existing roles to Scrum roles and achieve no more than relabeling of existing functions. Alternatively organization can define new roles description in alignment with Scrum framework, and prepare a new career path, to facilitate the transition of its people towards adoption of Agile behaviours and practices. By including its people in redefining its requirements and opening up new positions for interviews within the organization and also externally the organization can shift internal conversation from being victim of the changes happening faster and create bottom up momentum. This would allow the organization to grow strong agile mindset organically. Such experimentation at manageable scale, say a department or even a single-team can afford much learning at organizational level. Applying lessons learned from this experiment takes discipline and most importantly leadership courage to be able to face dysfunctions of past practices. No one knowingly hires dysfunctional people, so why in some organizations actions of many “good” people add up to a dysfunctional organizational behaviour? At some point the organization and its leaders have to take ownership at the systemic level. By enabling small, manageable, and rapid, experimentation the leadership can grow agility in alignment towards their larger strategic goals captured in Agile Strategy Map.

Focusing on customer value

Let me start by stating that I believe that “organization” is the set of structures and processes which govern the way people can collaborate - or not - with one another. As such I believe that organization can and should be changed to support people to work towards fulfilling specific objectives. So, if we consider the typical organizational structure, using for example a hierarchical structure, or even a matrix, we can in most of the cases observe that the main purpose of the organization is to optimize for efficiency. This means that what these types of organization are striving to achieve, if not enforcing, is the maximum utilization of resources. This approach, while probably valid in many market branches in 1911 when Frederick Winslow Taylor published the “Scientific Management Treaty” it is probably out of scope in 2016. In particular thanks to globalization and the internet infrastructure, many new business model emerged in the past 10 years, and many new companies entered markets which were before inaccessible to them. This change in dynamics, forced many firms to reconsider their setup, sometimes in a fight for survival situation.

I believe the market today demands organizations that are able to deliver what customers want, with high level of quality and satisfaction in a timely manner. In these market conditions, things such as the Cost of Delay (Donald Reinertsen) are playing a much more significant role, than the cost of development or production, thus organization need to change. An organization which supports value delivery, and time to market, needs to reduce to the minimum the amount of handovers as well as the waste in coordination and alignment. For this reason, most agile teams organize themselves as cross-functional and end-to-end teams, the same should happen at an organizational level. This has huge implications on existing structures and power plays, as everything which is coordination, synchronization and communication changes radically in favor of a more focused and value driven approach. For example, what happened at one of the organizations I am working with, counting several hundreds very specialized engineers, was to rethink the way the workflow developed by refocusing on the customer. This required identification of Value Streams to their customers, and also identification of that specialized engineers are delivering into these streams. It seems to be trivial question to answer, but believe me, more time than not, when we ask the question “Why are you doing what you are doing?” the faces in front of us look more puzzled than we would like them to be. Once the values are sorted out, or at least there is a strong opinion about those, we move on with what we call a Competence Mapping which can take quite some time, but aims at mapping the necessary competences to complete an increment of value.. The increment of value ends up being represented by what we call an Opportunity Canvas which should entail all the information that a given organization requires to make both investment and implementation decisions. It isn’t uncommon that once the Competence Mapping has been completed, involving normally a significant amount of people at different times, the results highlight that the very high level of specialization is hindering the organization to move towards smaller and self-organizing teams. We help with different strategies – for example establishing communities of practice, adopting pairing, creating expert teams… - aiming at broadening the individual competences and encouraging learning and growth. After some months we are normally assisting at what we call a Competence Alignment, meaning that the competence required to complete an increment of work, can be aligned with a smaller group of people, normally forming one self-organizing team. This is not always the case, and through subsequent adaptation (see the ETF explained above, with experimentation) the organization restructures itself, in the minimal amount of layers necessary to cover the required competences. Another very important step is to re-engage with the customers, and putting them back, close to the development of the product.

So if organizations want to be agile today - which means “to be able to turn on a dime, for a dime” - they need to invest in developing networked models fostering collaboration and independent delivery of value. This means that the power to make decision needs to be distributed into multiple networks to guarantee that reaction times will be such, that will not be constituting an impediment to value delivery. It isn’t uncommon in fact, that due to very long feedback loops on a traditional hierarchical organizational structure, people stop asking for permission, and instead they either wait and do what they are told to, or more rarely, they just go around the rules and policies to get their stuff done. When I work with organizations’ leadership teams, it is often the case - that the answer to the question “what is valuable and what is not?” is quite fuzzy. This is a symptom of a much deeper problem that is anchored to the fact that the culture of the organization has been pushed extremely towards executions and predictability, and processes and tools have taken the precedence over people and interactions. The assumption that the value proposition towards a market doesn’t have to change over time, is a major cause of company's failure or crisis.

Our approach is normally to challenge organizations to reconsider their assumptions and refocus on the essential value for each of their target groups. At one of our client, we helped the organization to rethink at their value proposition, by means of creating Business Opportunity Canvas which is one page document focusing mainly on the customer perspective on a specific problem which might reveal a business opportunity. The idea of having a single page that contains the original customer’s request, and evolves over time, instead of being replaced by different artefact is a powerful concept. Especially in very large organization is often almost impossible to go back to the original need, as it has been replaced by layers of generalizable solutions, out of which is very hard to regain control over the original context. Focusing on creating a single canvas is a very powerful process as it requires going through the whole decision making process, understanding all the steps involved and evaluating all of the risk on a single-page artifact. This approach brings all dimensions of opportunity to be considered for evaluation against other opportunities that are also vying of same organizational capacity. For every organization, it is an individual journey, that cannot be standardized.. So instead of developing yet another “canvas” we have developed processes and structures that organizations can use to create their own. This has a very strong impact on organizational acceptance and coherence with organization’s existing methods and standards. At this particular client, the creation of canvases, helped achieve two very important results:

  1. Transparency has been made on what the client needs are, and how decisions on what to pursue and what not are made;
  2. As a consequence of the first point, the way budgeting started being approached, changed from a yearly fight for the largest turf, to a continuous dynamic flow, where every idea is compared to whatever else is on the table at that moment in time, and gets planned, or postponed, or even tossed away.

Both of the results contributed immensely to help this organization become more agile, in fact decision making has never be more transparent and effective, and the organization is never been so flexible in changing direction.

As a second step, by focusing on each single Business Opportunity Canvas and developing the Minimal Marketable Scope for each of them, allows to start testing the idea with some pilot clients, before deciding to go full in with the full investment for development. Focusing on one opportunity at a time has the great advantage of allowing an organization to switch from planned or synchronized product releases to a more agile and continuous flow type of release process.

It is basically like extending the pull system towards the client, or the internal organization proxying it. If teams or small groups of teams were able to release features independently from one another, and integrate everything on a common stable stream, then business people could pick the content out of that stream and make a release at any time they want to. For example at another organization I worked with in the past, we have been able to have a “Customer Council” in place, who was challenged frequently with the direction the development of the product was taking. The Council contributed significantly in keeping the development focused on the most important things, and this was instrumental in removing waste from the process. So multiple teams worked in parallel on different Business Opportunity Canvases and by adopting a “toggle switch” system, they have been able to integrate their work on a common stable branch, but without actually activating it for the release, their code has been invisible to the clients till completed. This allowed for a significant flexibility in terms of decoupling the development process from the releases, and allowing every team to develop at their own sustainable pace. All teams – more than 100 to be more specific – have been integrating their work continuously, fulfilling a common Definition of Done and making sure that their features wouldn’t be deployed till completed (at least at a Minimal Marketable level). Using this approach, the organization has been able to engage clients with early releases, without having to wait for all other teams to complete their work. This in turn allowed to adjust very quickly to client’s changes, without having to wait too long for the next upcoming release.

This works under the assumption that everything which is integrated is working software. This approach allows to optimize for time to market, instead of efficiency. Another side effect of this is that by focusing on delivering increments of value faster, the amount of concurrent running projects within an organization decreases significantly, reducing dependencies and thus coordination effort. By releasing one feature at a time, we are also allowing the organization to decouple the decision to release, from the development cadence.

This also has interesting side effects such as the fact that the pressure on development teams will be significantly lower, or at least constant over time, instead of reflecting typical waterfall behaviors, where the stress increases exponentially towards the release date. Another drawback of organizations designed to optimize for efficiency, or capacity utilization, is that they tend to be always at the edge between capacity utilization and overburden. This situation doesn’t allow enough slack to inspect and adapt, and consequently leaves very little room for continuous improvement.

Decentralizing control

Decentralizing control allows for better agility because distributed part of an organization can react quickly and independently to changes. One of the goals that should be supported by the decision of becoming an agile organization should be achieving higher level of resilience or even anti-fragility. In organizations we coach, we help create models to delegate “power” by coaching leaders to design enabling constraints, which in turn encourages teams to self-organize. For example, you can create enabling constraints for a team, by sharing the goal they are to achieve. Followed by defining what “done” would mean to their customer, by defining policies that cannot be violated, and allowing teams to self-organize within this container. You will be surprised by the sense of purpose these enabling constraints can engender in motivating a team towards effectiveness. Delegating is a difficult process, requires building trust and requires transferring competences and knowledge. We work on both levels helping managers and executives to gain trust and delegate, by creating Containers for Empowerment that are built on trust premises. We have developed a framework to facilitate this process, which we call DECK. It is a metaphor for a sailing boat deck, which has to be kept tidy by the crew or the risk is something will go wrong, and even that someone will stumble and end up off board. It is an acronym for:

  • Direction: what are we trying to achieve by delegating this power? Why are we doing it?
  • Environment: in which context will this be valid? Will it be the team? The whole Product? The whole company?
  • Constraints: what are the constraints in place? What can’t be done? What needs to be escalated?
  • Knowledge: which type of know-how needs to be transferred between the two parties involved in order to successfully delegate?

This process is supported by Situational Leadership which allows move from a more directive approach, towards a more enabling approach. A DECK card is worked on collaboratively by the leader/manager together with the person or the team to whom a certain power/operation needs to be delegated to. By using this approach, leadership teams are gaining both more insights in what people are doing, and how they perceive the power, as well as actively building trust together with their people.

How to grow your own agility

First of all, think at agility as a means towards a goal and not the goal itself, I have seen of sort of self-fulfilling prophecies or crusades happening when becoming agile becomes the goal. Second keep in mind that changing an organization is definitely a challenge which belongs to the complex domain, and this requires it to be approached in an empirical way. In our Enterprise Transition Framework (ETF) we make use of the Cynefin Framework (D.J. Snowden) which suggests to approach the complex domain by means of safe-to-fail experiments in order to probe all possible options to improve/solve a situation, and allow for solutions to emerge. This means that companies should refrain the temptation of defining, and implementing, as it very rarely works. A typical reaction to a successful project is to try to Copy & Paste the processes and tools that people used while achieving that success. There is no guarantee though that they will work the same way with other people. The reasons for this are to be connected to the observation made at the beginning about Product Development not being a mechanically repeatable activity, but more of a creative and problem solving effort. Also, people require time to develop new skills and to learn use of new tools. What makes the difference though is that those tools are chosen by people who to fulfill a certain purpose, and aren’t imposed top down, by people who might hardly know how to do the job. The same applies to new organizational structures and processes. How often these things are changed to really improve the way people work? Besides if we want to have engaged people at work, we need to have them own the system of work, and imposing changes top down is going to cause resistance, and dissatisfaction.

Maybe one last piece of advice, don’t fall in love with pilot projects! Almost all pilot projects succeed! We need to be clear about what a Pilot project is, and what can be useful for. If an organization wants to try to implement Scrum and takes it to a team level, with the purpose of testing if Scrum works, well, yes it does. The real learning should be focusing more on understanding which level of impact the introduction of a system of work such as Scrum might have within an organization. In many cases people start working agile without telling it to their managers, because they wouldn’t get the approval. There is a very strong bottom-up drive to succeed, and people are willing to go the extra mile in order to make sure that it will succeed and make their lives better. This doesn’t mean that a similar setup will work on other people right away. The worst implication of this approach, are that companies which didn’t evolve their culture yet, are often making the mistake of copying and pasting processes and tools which made a specific team successful to the rest of the organization. This obviously doesn’t work very well, as the focus is again on the wrong side. Leadership team’s should be aware that from the moment they start “piloting” there has to be a strategic design to follow up on success or failure of the experiments, otherwise we would be disrespectful of the people volunteering to the pilots. Also it is way too late to start thinking about the implication of a successful pilot phase, once the pilots have proven to be successful. What is going to happen to process, roles, responsibilities, structures, training programs, career paths if the pilots are successful? So starting with a clear strategy and options is a better way to approach the evolution toward a more agile organization (see example with Agile Strategy Map above).

What to measure and how to measure agility

There has been may attempt to measure agility, but most of them fall short of capturing the real intent of it. There are even online assessment which rate the level of compliance to agile practices and disciplines, under the assumptions that if people use the right practices they will eventually become agile. There are also a variety of team performance models aiming at helping teams identify their level of trust, collaboration, and focus on results.

Ultimately I don’t think you can measure agility, and I would argue that it doesn’t make sense at all. On the other side, we can measure change and the fact that some of those changes actually produced better results than in the past. We can even use advanced tools to literally scan an organization’s culture and based on strategic design, observe if it is moving towards the behavior we want to amplify, rather than those we want to dampen.

As part of our Enterprise Transition Framework (ETF) we are coaching organization to assess themselves at regular interval against a set of indicators that are derived from the organization’s strategy. This allows to track the trends of evolution and determine if they are going in the right direction or not. The options are to either update their strategy to support the newly discovered direction, or to dampen the negative effects of the newly assessed trend. In a way makes it really concrete, and measurable, even if not really quantifiable, but is as good as we went, so far. For example with one organization we have been working with, there was a dominant hidden “assumption” that having specialized teams is a more effective approach to work. This happened because a group of very talented people, formed a specialized team and was able to produce good results, and was able to market them even better. So under this assumption the organization started a reorganization. We have been pulled in when the organization was experiencing major difficulties in completing their next product release, which had already been postponed by 4 months. The situation was very critical, the pressure very high, and mostly teams blaming each other for not having completed the work they were supposed to. Defects were bounced back and forth, with each team claiming that their design was right, and that the other team should fix the failure. All of this emerged only very late in the process when the “perfect” delivering teams needed to start integrating for the release. It was a difficult situation to come out from, but we managed to convince the organization to allow people to self-organize. We established some enabling constraints, and we asked volunteers to change the way of working and operating under the newly defined constraints. The most notable constraint was to be able to deliver working software into the mainstream every day, starting from the end of the first sprint, in 2 weeks time. The newly formed teams, focused on value, and used the team as a shield from external pressure, and a cradle in which to focus on getting things done. Very rapidly the results started emerging and the rate at which those teams managed to stabilize the product was just outpacing the speed at which the remaining teams were to some extent, probably, messing it up. A new strategy emerged as a consequence of the situation and the organization has been able to adapt very quickly. This subsequently lead to the whole product architecture being redesigned collaboratively, by the teams, bottom-up. This was a massive change also for the architects, who finally found fun in doing their job again, by working in the teams to support them deliver value faster. All the assumptions made about what was right and what was wrong, have been checked, and the leadership had enough courage to listen to the people doing the work, instead of following the plan. That organization is still happily evolving, every second of every minute of every hour, of every day.

If I may finish with a final suggestion from my heart -Do not focus at all on comparing the performance of an agile organization against another, nor comparing the performance of an agile organization against its own past. This mostly ends in a kind of witch hunting and silly vanity metrics identifications, which inevitably waste time and distract from the focus of the change. People’s, including clients and stakeholders, engagement and feedback are valuable indicators to determine the improvement. If people are happy at work, and are motivated, they are more productive, creative and focused, and very likely will achieve greater results. And finally, yes! there is a way to visualize agility, is called walk around the offices and count smiling faces and engaged conversations every day, if the trend is upwards you are getting there!

About the Authors

Andrea Tomasini one of the founders of agile42 and he has been working in the software development and product management as well as the process optimization arena for more than 20 years. He is one of the few people in the world owning both a Certified Enterprise Coach (CEC) and a Certified Scrum Trainer (CST) certifications.

Dhaval Panchal is a VP and Enterprise Agile Coach for agile42. Certified Enterprise Coach (CEC), Certified Scrum Trainer (CST), and Innovation Games Facilitator with 16+ years of experience working in the development and management of products and services in the software industry.

Rate this Article