Using OKRs to Build Autonomous Impact Teams

To focus on outcomes rather than outputs, Meilleurs Agents uses the Objective and Key Results framework to align the whole company on what they want to achieve. Christopher Parola and Nicolas Baron gave a presentation at FlowCon France 2019 where they showed how they implemented the OKR method and turned their product and tech teams into impact teams.

The Objective and Key Results (OKR) framework used at Meilleurs Agents is distinct from the original one in two main ways. They do not set OKRs for everyone in the organization, because they want everyone focused on what to achieve together, not individually; hence, they give OKRs to teams and not individuals. And they limit the number of KR to one per objective, because they want everyone in the organization to be able to memorize the three OKRs.

Impact teams are cross-functional, autonomous teams in charge of an OKR. Meilleurs Agents decided to go beyond the well-known concept of feature teams by assigning OKRs to these teams and measuring their success with it, as Parola and Baron explained:

We believe that a classic feature teams organization doesn’t guarantee that you work on the right ideas. You need to add clear, well-defined objectives to be sure you are looking to achieve an impact.

It might seem weird to say this in 2019 when a lot of people are talking about remote working, but having colocated teams helps to set up impact teams, as Parola explained:

Even though we are obviously not saying that remote is a bad idea, we have found a lot of value in having people in the same room and sitting at the same table, allowing them to adapt rapidly to our new organization.

"In our quest to help our developers be actors of the system and to find more meaning in what they do, our impact teams organization is definitely a success," Baron mentioned. "Our teams also have a better understanding of the platform metrics and it allows everyone to be a potential contributor and to suggest initiatives with an impact on our OKRs," he said.

InfoQ interviewed Christopher Parola, VP of product at Meilleurs Agents, and Nicolas Baron, CTO at Meilleurs Agents, about their experience with impact teams and OKRs.

InfoQ: What made you decide to implement OKRs?

Nicolas Baron: We were frustrated with our way of measuring team success. By using the number of delivered features as a metric of success, we weren’t always focused on delivering value and a lot of effort was going into shipping low-impact features. We were spending a lot of time crafting a 12-month roadmap which was out of date a month later. Teams were left wondering about the purpose of the features they were implementing. "Wait, why are we doing this again?"

As a CTO of a fast-growing company, I spend a lot of time building teams and hiring people. It made me a bit sad to realize that my people felt a lack of purpose in their daily tasks, and I believed we could involve them in a different way so they would feel heard and we could benefit from their ideas.

The conditions for an organizational change were also quite good. First of all, our CEO wanted to use OKRs to define objectives for the company. It was instrumental from a change management perspective, especially in order to rally the whole company behind that philosophy. Moreover, our teams were already quite mature on agile delivery methodologies as we had been using Scrum and Kanban for around two years.

InfoQ: How did you define and introduce OKRs at Meilleurs Agents?

Baron: The Meilleurs Agents Management committee sets the company-wide OKRs. We do that once per year and define a maximum of three Objectives with a given Key Result for each. We want to keep them as clear and simple as possible so that everyone in the company can explain them.

OKRs obviously require you to be data-driven and to make fact-based decisions. Consequently, we invested a lot on business analytics (people and tooling) and built a solid BI platform to help us on that journey.

Parola: To give you a concrete example, one of our objectives for 2019 is: "Meilleurs Agents is the go-to destination for real estate agents to attract sellers". The associated KR is "80% of our customers consider that Meilleurs Agents is an efficient solution to attract sellers".

Once the three OKRs are shared among the teams, the executive committee assigns them to someone who will be in charge of going through the process of gathering ideas from the whole company and prioritizing them. This person is not in charge of deciding what to do, but rather has to gather a lot of ideas. We are using the Impact Mapping methodology for this. On average we run three Impact Mapping meetings per OKRs to be sure that everyone in the organization - marketing, sales, management, product & development teams- can input the roadmap. You can be reassured that you’ll get a lot of ideas through this; we have around 70 suggestions for initiatives per OKR.

As we wanted to have a framework to help us talk about each idea, we use the RICE methodology (kudos to Intercom). RICE stands for Reach, Impact, Confidence, and Effort: (R + I + C) / E. It is like a value vs cost matrix, but with more levels. Its most interesting component is Confidence because it forces everyone in the company to deep dive into why they are convinced that their idea is good. It helps us decide whether we can start developing the product or if we need to run experiments to validate an idea.

For example, let’s take a new idea, like "we are going to rebuild our homepage in order to increase engagement". Reach will be the number of people visiting the homepage; in our case 200,000. Impact is a number between 0 to 3, which indicates how much impact we’ll have on the objective. We know the current homepage is already doing well; the impact could be 0,5 or 1. We have already built a lot of variations of our homepage and we know that it takes around 10 days (conception, design and development included). We ran a user test which showed already that people couldn’t find the agent directory from the homepage, so we know there is a problem, but we don’t have any clue how to fix it. We could go for 20% confidence in reaching the desired impact of this initiative. In the end, this idea scores (200000*0,5*20%)/10 = 2000. The score is useless by itself, but enables us to compare initiatives with each other!

InfoQ: How have you restructured your teams into impact teams?

Baron: The team structure is pretty similar to a feature team: it includes a product manager, UX designer, and front & back-end developers. We have around 8-10 people per team. As we want our teammates to be able to contribute to our OKRs with initiatives, we need them to understand the economics of our platform. And it is not easy. We are a two-sided platform operating on the real estate market. On the one hand, we have consumers (the B2C side) to whom we offer a 100% free service, and on the other hand, real estate agents (the B2B side), who actually pay for the service.

Understanding the mechanisms which make our users use our service takes time, and to help our people learn them as fast as possible, we make them switch from one team to another every 3 to 6 months. We call this the "Mercato". It accelerates a lot the knowledge sharing of our users’ specificities and helps our teammates to get a good overview of our platform. No solution is perfect and this one is no exception; you can feel the "cost of change" on your team velocity, which can sometimes drop a little right after the "Mercato".

Parola: When a product manager runs out of time, he focuses on writing User Stories instead of conducting User Research. It’s like a law of nature: an empty backlog feels like a failure! With impact teams, you have to be sure that product managers have the time to go through the Discovery process to improve your confidence level when deciding to prioritize an initiative. We also wanted to have the teams be as autonomous as possible, this is why we didn’t hire a User Researcher to do all the Discovery work, and instead gave this responsibility to our product managers.

To do so, we need two product managers per team; while one is focused on the Delivery, the second works on Discovery. We also took a lot of time to design our Discovery process in order to be sure that our product managers and product designers were fine with it.

InfoQ: What pre-requisites were there that made it possible to adapt the organization?

Baron: When you start to focus on the impact, you launch a lot of small experiments and only continue with the ones which positively impact your expected Key Result.

From a technical standpoint, there are important if not mandatory enablers to do this under good conditions:

• A solid automated test asset (unit tests, integration tests, UI, etc.). Launching a lot of live experiments on your platform requires speed to be effective. But you cannot break your existing features while you test new ideas. Automated testing remains the most efficient strategy to avoid it that I know of.
• A continuous deployment process and tooling. To be effective, the development cost of your experiments must remain as cheap as possible. As your team will increase a lot the deployment frequency, it is important that each deployment is as fast and secure as possible. To do so, we rely on pretty common tools such as Jenkins, Terraform and Ansible, and all developers can deploy whenever they need to.
• The last one is not mandatory but surely helps a lot. Our technical stack, both on the B2B and B2C side, is mostly homogeneous. The cost of the "Mercato" we’ve talked about before is, therefore, less important.

Parola: As Nicolas said, we were at our 50th sprint. Agile methodology wasn’t a problem anymore and teams were able to deliver value by leveraging our tech stack and the organization already in place. We also had product managers who were able to run User Research and quickly test a lot of ideas. These are important ingredients in taking a step further and in assigning OKRs to your teams.

InfoQ: How do you deal with technical debt?

Parola: With our RICE scoring, we can see if an idea is risky or not. If this idea is risky and expensive to build, we will work hard to come up with a better idea if it is worth it.

Testing fast doesn’t mean doing crappy code; no one ever learned anything from a broken test. But as we want to test a lot of things and are not sure what will work, we need to be faster (less automated testing, less pixel perfect design) in order to know if we will be keeping a feature or not. It’s really hard because developers, especially the senior ones, already know what could happen. "The current version is already working well, why do you want to invest three more months to make it perfect?!" We had to be very clear on two things with regards to handling those crappy tests: how to help the company understand why we need to rework something, and reassure developers that they will have time to improve their code after the testing period.

Baron: What Christopher describes is the definition of debt in its purest form. It would be very sad to develop something perfectly that doesn’t match our user needs or achieve the impact we want. So we contract a technical debt to invest in our experiments and develop them quickly, with potential trade-offs on architecture, tests coverage or quality of code.
To keep a healthy system, once our experiments are done, we reimburse it. In my opinion, it is very important to see this as an investment strategy where we generate debt in the short term but lower the waste - useless code and/or features - in the long term.

InfoQ: What have you learned on your journey with OKRs?

Parola: Imagine that you have an almost infinite roadmap every year, with the capacity to fill it with almost every need you have, and all of a sudden, you are told to stop and you can choose only three OKRs. It’s hard, right? It’s what happened to management by the end of 2017. In fact, It was so hard that we ended up setting seven OKRs instead of three. That’s actually more than the number of business units in our company. At the time, we only had two impact teams up and running. It led us to unpleasant conversations every quarter, basically fighting about which OKR was the right one to work on. As we didn’t choose early, we had to choose frequently. Fortunately, we improved this in late 2018 and ended up with only three OKRs: one for the B2B, one for the B2C and one company-wide. Decision-making became a lot easier with less OKRs.

Anyway, for our 2019 OKRs, we had to wait four months to measure our Key Results because we had chosen high-level metrics (for example customer satisfaction, measured by a polling institute). We are currently changing this, thanks to the help of the BI platform Nicolas talked about, into more actionable metrics like the percentage of conversion on the platform. We hope it will help teams to measure their direct impact on the OKR day-by-day which is very important in order to keep everyone focused and to adapt quickly.

We also ran into trouble a few months after the implementation of OKRs. Our teams were so happy to be granted so much autonomy that they forgot to listen to management’s ideas and to frequently communicate about their progress. Management was blind on what the teams were doing, and this sudden feeling of losing control made us think about a rollback to our previous organization. Fortunately, I read an article from Netflix’s VP of product in which he explains that he does roadmap meetings every six weeks to give visibility on the past weeks’ main achievements. We tried this and it worked like a charm. Six weeks is long enough to have an impact, and short enough to remember why we are doing what we do. We also included top management in the Impact Mapping meetings to gather their ideas and score them shortly after using the RICE framework.

Baron: The "Mercato", by frequently mixing up our teams, had worked really great for almost two years and helped to share knowledge and to give everyone an overview on the key mechanisms of our platform. Anyway, as we have significantly bigger teams now (we’ve more than doubled during the last 18 months), the cost of our "Mercato" has become more important. As we now have one engineering manager per team, it’s also been complicated for them because they often have to manage people who are part of their team for three months and then move to another team.

So, for 2020, we decided to adapt it: developers can still switch to a new team but the frequency is less important.

Two months ago, we were acquired by the AVIV group, a subsidiary of Axel Springer. And we also keep growing our teams. In this brand new context, our objective remains exactly the same: to keep a high innovation pace. We believe our Impact teams organization will be a strong asset in keeping this up.

About the Interviewees

Nicolas Baron is CTO at Meilleurs Agents, a real estate platform leveraging data & science to connect sellers & buyers with real estate agents. He's passionate about building impactful products and scaling tech & product teams (engineering management, team coaching & culture, lean product & development practices, complex platform architecture) for fast-growing startups. Happy father of two, and a passionate runner.

Christopher Parola is VP product at Meilleurs Agents. He's passionate about building the right product team, process and organization to achieve high impact and deliver great team products. Always looking for better ways of working with the whole organization and achieving high velocity hand-in-hand with tech teams. Happy father of one.