IDEA: a Framework for Nurturing a Culture of Continuous Experimentation

"The illiterate are not those who can’t read or write, but those who can’t learn, unlearn, and relearn."
- Alvin Toffler

In a world of constant change, competitive advantage goes to those who can adapt and evolve. Learning is part of adaptation. Learning requires change. For the past couple of decades, agile practices have become common in software development for this very reason.

Some may argue that "agile has failed". I will argue that agile has not failed, but our agile practices have not evolved with time. Agile is like music, and just like music has several genres, there are many different agile methodologies. Agility, on the other hand, is not bound to any specific methodology. Agility is how fast we can respond to change. For a team to be agile, they need a culture that allows them to learn, unlearn, and relearn. The team needs to embrace change and be trusted and supported in their learning process.

In this article, I will explain how teams can foster such a culture, navigate through the complexities of modern development environments and harness agility to deliver software quickly that fits the needs of users and business sponsors. I will introduce the IDEA  (Imagine, Decide, Execute, Assess) framework that helps form the foundational layer of project initiation and development. This framework will allow teams to explore, plan, implement and evaluate ideas. Once a project is underway, after a Minimum Lovable Product (MLP - a basic product, feature, or a design that sparks user delight and anticipation) is achieved, iterative learning models such as PDSA (Plan-Do-Study-Act; aka The Deming Cycle) become relevant, guiding teams in refining and improving processes for ongoing development and success. Together, these methodologies create a comprehensive blueprint from ideation to continual improvement.

Defining Agility

 Agility is the ability of individuals and teams to change the product or process based on feedback or new information. Agility requires a continuous cycle of learning, unlearning outdated methods, and relearning new skills, technology, or information. Effective agile methodologies enable and provide structure for these changes.

The power of agility is not limited to managing change; it’s also about creating a framework that enables exploration with limited downside risk, sparks innovation, and drives business value.

The IDEA Approach

I came up with the IDEA approach several years ago for experimental projects, though back then it didn’t have a name. I had found the cycle of asking a question, deciding how to tackle it, building the Minimum Viable Product (MVP) and iterating on the feedback to make it worth releasing to the users very valuable. The approach empowers teams to evaluate if a potential project is a good fit for investment of time and resources. Since exploratory projects often have limited time and budget, IDEA provides a mechanism to confirm if the project is worth more investment. Most importantly, the framework guides the development of a MLP. The MLP starts once an MVP is achieved. This is an evidence-based approach. Feedback gathered in the real world from stakeholders, initial users, and the system itself are essential to go from a MVP to a MLP:

• Imagine: This phase is all about exploring new ideas with an open and creative mind. Here, the team identifies the next best test and considers questions such as the business or user problems to be solved, profit opportunities, cost benefits, and skills required to explore the potential project.
• Decide: After agreeing that the idea is worth exploring, the team needs to decide on the scope, setting delivery goals, outlining a high-level strategy to tackle the problem, and defining what the MVP or the MLP looks like. The team should also assess the realism of the idea, the potential challenges, and the ideal exploratory team structure.
• Execute: It’s action time! With a defined goal and enough information to get started, the team proceeds to execute the project. Key elements in this phase include task assignment, development, stakeholder communication, and expectation adjustment. Regular checks are necessary to confirm the progress is being made towards the goal, the team is on the right track, the necessary team members, skills, and tools are in place, and that the project goals remain realistic.
• Assess: In this phase, the team collects feedback from stakeholders, users, and the system to build the MLP. Feedback is invaluable in making informed decisions on whether to continue building and adding features, refining the idea, starting from scratch, or even terminating the project. Questions to consider during this phase include what has been learned from the feedback and where the project should go next.

The IDEA framework, coupled with the "Nail it, then scale it" philosophy provides a potent strategy for exploratory projects. The "Nail it, then scale it" philosophy underscores the importance of perfecting an idea or solution before scaling it up, thereby ensuring its viability and relevance at a larger scale.

Here are some specific suggestions to make IDEA work well, grouped into topics.

Explore New Ideas

Coming up with new ideas involves a combination of creative thinking, strategic planning, collaboration, and openness to change and learn.

For example, if you need to come up with a new product idea, consider inviting all relevant parties - stakeholders, executives, team members and select users or customers - to the workshop in which:

• Everyone is free to put their ideas on the board (whiteboard, miro board, etc)
• They get to present their idea to the group in 1 min or less
• Then regroup the ideas (find those that are very similar to each other)
• Allow members to go back to the board and vote on ideas they want to explore more (let’s say everyone gets three votes and they can use it however they want, including vote three times on their own idea)
• Next is to find the most popular idea(s), form an exploratory team around it and get things started!

Make Good Scoping Decisions

Here are a few tools and techniques that I’ve found to be effective:

• MVP in terms of features:
  • Ask all the features to be written down
  • Prioritized in terms of their business value and complexity to build them, with the highest valued features having the lowest complexity on top of the list
  • Start taking features out, until the product no longer exists without that feature
  • Then put that last feature back in the list
  • Start building from the top of the list
• MVP in terms of time, this one is from Jeff Patton’s "Passionate Product Leadership" workshop:
  • Put all the features on the board
  • Sort them in the order of their dependencies to each other (or by time)
  • Next imagine you only have 6 months to build something. Which features do you take out?
  • What if we have 3 months? What if you have 1.5 months? ...
  • Take the features and put them on a "high perceived business value / low complexity to build" grid
  • Start with features that are in the first grid (highest value and lowest complexity)

Foster the Culture of Continuous Learning and Experimentation

As a leader and a consultant, I am used to embracing the unknown - I may not know something, but I have a process to find it out. That helps the team understand that they don’t need to have the answers to everything, but they must be flexible to pick up new skills, find resources, and be open to learning new things.

Empathy and trust goes a long way when building relationships. If the team is expected to pick up new skills, they need to have dedicated and uninterrupted time to practice and learn. As a team, you can timebox the uninterrupted time you need. However, expecting your team to pick up new skills while they’re also expected to work full-time on their current projects will end up in disappointment and burnout.

Another important factor is that people adopt new skills differently. Some people learn better in groups and some alone. I always respect individual preferences. However, having a couple hours of workshops for the whole team often benefits everyone. During these workshops everyone can discuss their learning, questions, and interesting facts they found out.

From my experience as a consultant, I often find myself stepping into the unknown with new clients and projects. This has taught me that openness, honesty and curiosity are fundamental. In my consulting engagements, a culture of continuous learning is not merely a theoretical concept but a practical and integral part of our work which makes each project a new adventure and every challenge a new opportunity for growth.

Moreover, part of fostering a culture of continuous learning and experimentation involves strategically anticipating potential challenges. This is where conducting pre-mortems are instrumental. The pre-mortem allows the team to talk through possible obstacles or failures before they occur. Obviously, we cannot account for every possible risk, but conducting pre-mortems has many benefits. One is to help teams to develop proactive strategies and solutions, thereby contributing to a more resilient and adaptive environment. Another benefit is that the team is more prepared to handle a potential failure when and if it occurs because they’re mentally more prepared.

Balance Exploration and Exploitation

Understanding a team's capacity and their existing skills are important factors in striking the right balance between exploration and exploitation. Introducing more than one new concept at a time can dilute focus and hinder effective learning. Moreover, it’s essential for the team to have the opportunity to master new skills and gain confidence in applying them. Once the new skills are achieved, celebrate them to boost team morale.

However, constant change can cause the team to feel unstable and eventually burn out. Balancing exploration and exploitation is specific to each team and should be adjusted based on their comfort and interest.

Collect Useful Feedback

Depending on the project, you may want to collect feedback from several sources. That includes stakeholders, internal users, external users (customers), and the system. For example:

• Collecting feedback from your stakeholders could be done during regular demos (hopefully no longer than bi-weekly)
• Collecting feedback from internal or external users could be done through frequent deployment to the production to allow them to test the new features
• Collecting feedback from the system could be done by putting logs and metrics in right places to collect relevant data that helps you make decisions

When you are collecting feedback, especially from the stakeholders and users (internal and/or external) diversification is key. How much diversification do you need? We may not have the capacity to include everyone and there’s always a chance of leaving someone important out. When it comes to sample sizing, consider those who are the majority of the users, as well as those who are the risky samples even if there are only a handful of cases.

Furthermore, it’s important to choose appropriate metrics for evaluation. These could range from usability rating, system performance, customer satisfaction score, drop-out rate, etc. Collecting the right metrics ensures we capture relevant data to inform strategic decisions. We want to make decisions with the lights on, not just based on our gut feelings.

Use Feedback for Decision-Making

Feedback serves as a guidance on the development journey whether it’s related to adding new features, enhancing existing ones, or even pivoting the direction of the project.

During the Decide phase (deciding on the scope of the project), it’s crucial to stay focused on validation of the core concept, hence, additional features should be deferred unless they prove crucial for the Minimum Viable Product (MVP) or Minimum Lovable Product (MLP). This practice is going to help the team focus on building features that the stakeholder can use to decide if they want to persist or pivot.

During the Execute phase (the product development), feedback can significantly impact priorities. Every new feature or enhancement request should be measured against what’s in the scope to prevent scope creep. Consider if the new request has a business value that fits in the MVP/MLP, and the complexity to deliver it. That said, all ideas are welcome and they can be added to the backlog until the time is right to implement them.

Once MVP is achieved, the aim should be to transform MVP into MLP. This could be achieved by incorporating users’ feedback, considering some features that make the users love the product, and build the one that’s highest in value but lowest in complexity to build. This iterative process of applying feedback ensures that the product continuously improves and evolves in response to the users needs and preferences.

On the other hand, if the feedback is scarce, it could indicate that the product isn’t engaging users as expected or perhaps the methods of collecting feedback are not effective. Here are a couple of examples:

• Proactive feedback collection: if the users are not voluntarily providing feedback, try proactive methods such as:
  • Surveys
  • User interviews
  • Usability tests
• Analyze usage data: analyze usage feedback to collect information on what features the users are most interested in
• And ultimately, if the pivot is necessary:
• Evaluate: use the information you have to identify what’s not working and a potential new direction
• Revisit the initial question (Imagine): Based on the experiments and the feedback received, make a decision to tweak the initial question or document the learning and terminate the project

It’s very important to note that pivot is not a sign of failure, but an opportunity to learn. It demonstrates adaptability and a willingness to build the right product for the user. Learning from the feedback is a key attribute of a continuous learning and experimentation culture. This is also why on exploratory projects, it’s important to start small (in terms of team, budget, and features) until you have some metrics that show there’s an opportunity to grow.

Lessons from Using the IDEA Approach

Using the IDEA approach helps me with valuable insights into the potential of various projects. In addition to deciding whether or not to pursue an idea, the process helps the team understand what resources, skills, and efforts are required. Iterating through Imagine, Decide, Execute, and Assess helps continuously learn and adjust our strategy based on real-world feedback. The IDEA framework, complemented by the "Nail it, then scale it" approach empowers the team to traverse the landscape of uncertainty with confidence. That also allows the stakeholders to make informed decisions about scaling the team and investing time and budget into the project. Informed decision-making increases the likelihood of a project’s success and minimizes the risk of resource misallocation.

Navigating the complex dynamics of the modern software ecosystem calls for more than technical prowess; it requires a mindset shift, a cultural transformation that nurtures experimentation and iterative improvement. Each step in this iterative process brings us closer to realizing the collective potential and delivering solutions that are not just viable but lovable, not just effective but transformative.

Agility is about moving forward, and every step counts!