BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles DevOps is Not Enough for Scaling and Evolving Tech-Driven Organizations: a Q&A with Eduardo da Silva

DevOps is Not Enough for Scaling and Evolving Tech-Driven Organizations: a Q&A with Eduardo da Silva

Bookmarks

Key Takeaways

  • Organizations are continuously changing and evolving (contrary to many mainstream models that target static structures).
  • Evolution may be triggered by different aspects, external such as customers and market, but also internal such as the organization structure or technical architecture.
  • DevOps has been a great evolution step to have more cohesion and alignment on development and operations, however organization systems (building a product) have other elements (parts) that also must be understood and considered to better enable our teams for "fast flow".
  • Approaching evolution with an holistic sociotechnical systems and architecture perspective provides for a more comprehensive understanding and co-design of organization and technical systems.
  • Data science is a great example of having different types of approaches as the organization grows in scale and complexity (from project-based approach, to "data science teams" owning key products, to actually treating data science as a discipline part of the multi-disciplinary team that builds and evolves a product continuously).

Eduardo da Silva, principal tech lead & sociotechnical architect at bol.com, the largest online retailer in The Netherlands and Belgium with 12 million active customers, recently spoke about the need for a sociotechnical systems thinking approach at the DevOps Lisbon meetup. Silva illustrated how DevOps was a good starting point but a wider view of the organization as a sociotechnical system was key for sustained growth and learning at bol.com over the last decade.

InfoQ reached out to Silva to explore these ideas and understand how they supported bol.com’s evolution.

InfoQ: What does system thinking mean in practice at bol.com and why has it been important for sustained growth in the last decade?

Eduardo da Silva: Systems thinking is being adopted more explicitly at bol.com over the last few years. This does seem like a recurrent pattern in many organizations that want to grow and be able to scale their abilities. That is not easily achievable and systems thinking is key to support that, namely to have a better understanding of the overall system (its parts and interactions) and from there define more appropriate evolution strategies. At bol.com these practices have been crucial to support strategic changes on the company sociotechnical architecture, e.g.: growing from being a web shop with its own stock to a retailer platform where many sellers can sell their products; and more recently becoming a product-led organization (where business and tech work more closely together). Figure 1 provides an overview of key elements of this evolution. From this picture it is clear that product, organization and tech architecture tend to co-evolve with each other.

Figure 1: bol.com evolution from 1999 to 2019

InfoQ: You mentioned in your talk that we need “BizDevOps”. Why isn’t DevOps enough?

Silva: DevOps has been an evolution of breaking silos between Development and Operations to enable technical teams to be more effective in their work. However, in most organizations we still have other silos, namely: Business (Product) and IT (Tech). "BizDevOps" can be seen as an evolution from DevOps, where the two classical big silos in organizations are broken into having teams with the product and tech disciplines needed to build a product. This evolution is happening in many organizations, most of the times these are called "Product Teams". Is it enough to maximize impact as an organization? I don't think so, and that is the focus of my DevOps Lisbon Meetup talk and ideas around sociotechnical architecture and systems thinking I have been exploring. In a nutshell: we need empowered product teams, but teams must be properly aligned with value streams, which in turn must be aligned to maximize the value exchange with the customer. To accomplish this, we need to have a more holistic view and co-design of the organization structures and technical architecture.

InfoQ: Despite most services and products being tech-driven these days, that business (or product) vs tech divide is still pervasive in many organizations. What do you think are the main challenges: lack of adequate organizational structures, lack of common language, lack of business domain understanding, or just historical baggage?

Silva: This varies per organization, I have seen all of those elements playing a role on this divide. I think the strongest one is the "historical baggage" (and culture): organizations tend to get into habits and routines, e.g.: having business and tech silos. This is a rather pervasive model in industry, and in fact follows a pervasive modern thinking model of breaking down systems into parts and optimizing parts in isolation (this is analytical thinking, the exact opposite to systems thinking). When you combine that with another common trait of striving for "fixed org structures", i.e.: neglecting that sociotechnical systems are in continuous change, this becomes an even bigger challenge. So, to me the key is to have a systems approach and also embrace change and evolution as key design principles for organizational operating models. For example: the evolution of DevOps to "BizDevOps", where product disciplines work closely with tech disciplines is a natural evolution to make sure we can build better products at higher velocity.

InfoQ: Did bol.com experience some of those challenges and how were they tackled? Were there any particular events that accelerated the need for business and tech areas of the organization to align and/or merge?

Silva: Yes, most definitely. Figure 1 shows key aspects of the sociotechnical architecture evolution of bol.com. You can see how certain product and business events trigger sociotechnical changes. For example, around 2012, after the takeover by Ahol, bol.com moved from being a web shop with its own stock to being a retailer platform where many sellers can sell through the platform. That event required a lot of innovation and development of new offerings. This led to the need to evolve to a more flexible and scalable technical architecture, namely a (micro)services oriented architecture. The reason for this was that we knew that we needed to have many more people/teams working on many different parts of the landscape and be able to independently release their work at high velocity. To achieve that, and coming from a classical "waterfall way of working and organization" (with monthly releases of monoliths), we had to transform our organizational setup. This was the trigger for adopting DevOps. This sociotechnical evolution was reasonable then, but things continue evolving. Fast forward about 7-8 years, the company grew from 30 teams to about 100 teams and that model, where each team owns specific services that form capabilities on the landscape, was not scaling well anymore. So, again we had an evolution, in this case becoming a product-led organization. In this evolution, business and tech start working even closer with each other.

InfoQ: What is sociotechnical architecture for you and what part did it play in bol.com’s growth so far? Could you give a concrete example?

Silva: My definition of sociotechnical architecture is: “taking an holistic co-design approach to technical and organizational systems, given the inherent impact they have on each other”. Figure 2 shows my overview of the main elements and relationships of sociotechnical architecture. At bol.com, this co-design has been key to enable the company to evolve and scale. Apart from the examples mentioned earlier (e.g.: becoming a retailer platform, and later a product-led organization), there is another very interesting one: the introduction and evolution of data science in the organization. Data science was introduced in the company around 2012 as a way to scale and manage complexity on certain parts of the landscape (e.g.: customer recommendations, sales forecasting, etc). In the subsequent years we saw a series of evolution iterations that enabled data science to align with the sociotechnical architecture of the company.

Figure 2: main elements and relationships of a sociotechnical architecture

InfoQ: Is it fair to say that data science started at bol.com as a (niche) business need met by specialized delivery team(s) and evolved towards an organization-wide capability met by supporting teams and structures like platform services and experts in enabling roles?

Silva: Yes, that is true at bol.com and I think in most companies. At bol.com data science was introduced to support strategic (niche) parts of the business. Given that it was mostly targeted to support certain parts of the landscape, and also we had very few data scientists, this iteration of data science was project-driven. Basically data scientists would be implementing solutions in different domains and teams in the company. 

However, the company continued to grow extremely fast and more and more products needed this capability. Furthermore, this project-based model was not aligned with our strong DevOps culture ("You Build It, You Run It, You Love It"). This led to the second iteration, where we created a data science department with a two-fold mission: create and support dedicated "data science teams" who take full ownership of their products and also develop the overall data science capability (people/organizational and technical platform to do data science). This step addressed some concerns and enabled us to learn and develop a lot of elements to mature data science in the company. 

But this was still not aligned with the organizational architecture, where products are organized around values streams to the customer, and manage the disciplines necessary to successfully build and evolve the products. This led to the following evolution iteration, namely: consolidate data science as a "discipline" that can be used on our products. In practical terms this meant products become responsible for hiring and managing data scientists, while there is an "enabling team" of "Data Science Craft Leads" who continues developing and consolidating the discipline (and its community) and also helps in its adoption into new products. 

This consolidation makes data science a discipline like any other disciplines within the multi-disciplinary teams in the products (of our product-led organization). Figure 3 shows an overview of the sociotechnical architecture evolution of data science at bol.com over the years using my sociotechnical architecture modeling representation.

Figure 3: sociotechnical architecture evolution of data science at bol.com

InfoQ: Do you think that we need to evolve sociotechnical systems by regularly sensing if the current organizational structures are promoting or impeding the desired business outcomes and required technical capabilities?

Silva: Yes, I think that is key. I would in fact say we need to continuously sense the different parts of the sociotechnical architecture and make sure they are not at “odds" (as Ruth Malan says). This can be achieved in different forms, e.g.: track Accelerate metrics, measure teams cognitive load (or team health), etc. We want to have continuous feedback loops to sense the sociotechnical architecture. With this we are continuously learning how the different parts of the system are and with that form an holistic understanding of the system, from which we can drive its evolution. 

For example, if we observe a team is struggling to go through the product development cycle (e.g.: lead time is too high), we get a trigger to look at what is happening. This can be caused by issues on the technical architecture, e.g.: tech debt, but may also be just a fact that the product became too complex to be owned by the team (i.e.: team is reaching its maximal cognitive load). This understanding can only be attained if we have feedback loops and data on all of these elements, i.e.: we should not simply add more people or teams, this will not solve anything if the issue is a poor evolution of the technical architecture. 

To implement this, especially in large scale organizations (such as bol.com), we also need to make sure we have people interpreting these feedback loops at different levels. For example, at bol.com we have "structural enabling teams'' in the form of product managers, engineering managers and tech leads, who look at the different aspects of the sociotechnical architecture (product, people and tech architecture). These enabling teams also form networks of knowledge (across products), which allow to sense the challenges observed on the different products and with that understand how the organization is doing as a whole. 

So, basically different levels of systems but they are interrelated and in continuous evolution. To accomplish this we need to embrace "continuous evolution", where sensing and holistically co-designing the sociotechnical architecture becomes our new organizational operating model.

InfoQ: Last question, what do French restaurants have to do with all this?

Silva: Great question! Systems thinking and sociotechnical architecture are topics that are not commonly used (yet). This has multiple reasons, but the most important one is our inclination (and training) to break and think in parts (analytical thinking). Given this, I have decided to motivate this theme using the metaphor of a restaurant. Restaurant is the system, which we want to optimize as a whole. People understand that we are not interested in purely improving a dish if it does not provide the best experience to the customer. We want to make sure all the parts of the restaurant are taken into consideration when designing the best experiences for the customers. That's why I keep on using the "French Restaurant" metaphor. Furthermore, it is fun talking about restaurants at technical conferences and discussions.

About the Interviewee

Eduardo da Silva is an independent consultant in sociotechnical systems evolution, architecture, and leadership. He also is a Team Topologies Valued Practitioner (TTVP). His work focuses on helping tech-enabled organizations find strategies, structures, and operating models to achieve and scale a sustainable fast flow of change. He takes a sociotechnical systems’ framing, considering org/people, technical, product (customer and environment), and business perspectives. He regularly speaks at events and writes about these topics at esilva.net Since 2005 he has been an academic researcher, entrepreneur (founder of a startup), software engineer/architect, and consultant in multiple companies (from startups to scale-ups). From 2017 to 2022, he was a principal tech lead and sociotechnical architect for the largest online retailer in The Netherlands and Belgium (bol.com). He worked on various technical and organizational challenges to scale this 150+ team product-led organization.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT