David Stein, principal AI engineer at ServiceTitan, presented Moving Mountains: Migrating Legacy Code in Weeks instead of Years at the very first QCon AI New York 2025.
Stein kicked off his presentation by describing how large-scale migrations are synonymous with "moving mountains." Major architectural changes have historically required time measured in quarters or years, along with inherent risk in rewriting large amounts of code. While the benefits of a successful migration may be significant, the risks may be equally substantial because migrations may not always work out as planned.
He presented an internal case study on ServiceTitan, which migrated its legacy reporting datasets to its new metric platform based on DBT Labs MetricFlow. This included: a reliance on production SQL databases; hundreds of metrics calculated in a monolith application using object-relational mapping; and thousands of lines of C# and complex SQL code. The timeline was quarters to years.
Stein introduced the term "false summit," a concept related to hazards that may occur during a migration that may lead to failure. The hard parts of the migration may sometimes stall, and those benefits are never realized. Developers are left to ask, "Wait, are we building the right thing?"
The Principle of Acceleration is a concept involving techniques to boost performance by optimizing data flow. As shown below, the old way involved a decision point early in the process, after establishing initial proofs of concept, before investing in the migration. The new way, however, adds a validation step with a standardized set of tools used before the migration begins.
The Assembly Line Pattern, a sequential, automated process for building, testing, and deploying applications, often used in CI/CD, is composed of three steps in transforming an application migration:
- Decompose: break the "mountain" into "pebbles."
- Standardize: create a "physics engine" for automated and instant validation.
- Automate: use of agents to execute tasks.
Stein listed the breakdown of tasks used by ServiceTitan for their heritage migrations, namely: objective, plan, code, test, and release, which took them years to complete. Using the Assembly Line Pattern, they can now execute a successful migration in a number of weeks.
Stein discussed a number of challenges with their corresponding solutions along the way:
- The agent incorrectly believes it has succeeded → fix the validation logic.
- The agent gets stuck and can't make progress → add context and test data; subdivide tasks.
- Small portions of tasks can't be completed → engineers manually complete the most difficult tasks.
- A better architecture is identified partway through → rewrite the migration, goals, and validation; start over.
The paradigm shift in reducing the time to change architecture from years to weeks unlocks a new level of agility. Stein mentioned that this has enabled developers to ask themselves new questions for which he offered suggestions that required only weeks to implement:
- "Should we try platform X?" → Just do it and find out.
- "Is this new architecture actually better?" → Build both and compare.
- "Did we make the right call two years ago?" → Remigrate and check.
Stein concluded his presentation by stating that migrations aren't "mountains" anymore.