Travelopia changed its focus from a technology approach to business outcomes, and adapted agile and lean for delivering machine learning solutions. This enabled them to deliver machine-learning business models faster and better.
Sreekandh Balakrishnan, "Director - Innovation" for Travelopia, and Simon Case, head of data for Equal Experts, spoke about machine learning at Travelopedia at Lean Agile Scotland 2022.
The first iteration of machine learning was very technology-focused, Balakrishnan mentioned:
We took a technology stack approach and built a Data Lake before understanding the business use-cases. We took a big bang approach to delivery, a large team, with a promise to deliver multiple use-cases once the data lake was in place. At the end of 18 months we had 3 models in production but no less business unit using it.
Case suggested to understand what business improvement you want first:
Start small, pick a slice of value and learn not just about the tech, but also how to frame the problem, how to get users onboard, how to provide the results to the users or downstream services, and what other organisational challenges (people, process first & finally tech) are there.
Balakrishnan mentioned that they were not making either the technical or business impact that they wanted. They made some changes to become leaner and changed their focus to business outcomes:
We had a second iteration using the new lean, agile approach and we were able to deliver 2 models in 3 months which are being used and creating business value. After this success, we adopted this as our methodology. We have since scaled to 10 models in production for 5 brands. All are being used in the business and some brands are generating close 21% incremental business wi. In fact, I can now deliver a new model into the business in under 10 weeks.
InfoQ interviewed Sreekandh Balakrishnan and Simon Case about machine learning at Travelopedia.
InfoQ: What did you learn from how you initially applied machine learning, and how did that impact your approach?
Sreekandh Balakrishnan: We applied lean, agile principles - find out what is valuable, deliver in small increments, and keep learning (and pivoting until you get it right).
We changed the focus from a technology to business outcome. We took the time to understand what the business wanted and how they wanted it delivered. The team was too big, so we reduced it from 40 people to a team of 6. We found that a lean, cross-functional team was able to make progress faster and keep the focus on what the business wanted.
We stopped creating a data lake which would serve all ML needs, and started focusing on just getting the data ready that we needed. This also had the side effect of reducing our cloud cost to 10% of what it had been.
We also made a conscious pivot away from GUI driven tools. We had used them in the first iteration, but found it hard to apply modern software development techniques (TDD, Pair programming) when using them. Rather than accelerating our delivery, they were slowing it down.
Finally, I recognised that I needed buy-in from the business, who were more used to big bang delivery. So I made sure I had an exec sponsor who understood and was bought into this approach. This really helped our relationships with the business, and made adoption of the ML models easier.
InfoQ: In your talk, you suggested not to worry about data platforms, but instead worry about how your teams will self-organise. Can you elaborate why?
Simon Case: Machine learning is a team sport. You need the data scientist and the data engineers to work together. They are different disciplines - data scientists are skilled at algorithms and mathematics, but often do not have the software skills needed to make reliable products. The temptation can be to keep them apart, but if you do this they won’t learn the techniques needed to put their work into production.
Balakrishnan: Things started to change for us when we slimmed the team down and created a single small cross-functional team. With the data engineers, data scientist and business analyst working as one team, they understood the users better and could make rapid decisions and trade-offs on the technology stack and the ML model.
InfoQ: What’s your advice to companies that are considering using machine learning?
Balakrishnan: Start small, and don’t worry about buzzwords! Keep business stakeholders close to you and invite them to join daily standup/planning meetings. Create the first win and go to production. Remember, it’s a learning for you, your team and your business.
Case: Build your competence iteratively - start with a steel thread - the first vertical slice which provides useful business value. Use this to learn quickly, and when you are happy with it, it acts as a pattern you can use for other ML models. If you’re lucky, it becomes a paved road - a way of working that allows you to quickly deliver new models into the business.