Q&A with Susan Fowler on Production-Ready Microservices
The Microservices.com Practitioners Summit is a practitioner-driven microservices conference that focuses on real-world applications of adopting microservices at scale. The Summit will be held in San Francisco on January 31, 2017. Speakers include microservices practitioners from Uber, New Relic, Lyft, PayPal, and Google.
Susan Fowler is an engineer at Stripe, the author of Production-Ready Microservices, and will be speaking at the Summit. While at Uber, Fowler worked on microservices standardization, as well as embedded within various business-critical teams to help make their services more reliable.
Prior to the summit, InfoQ met with Fowler to discuss the technical, business and cultural challenges of successfully implementing a microservices architecture.
InfoQ: Where do you see the ideals of a microservices architecture intersecting, and possibly conflicting, with the real-world systems at most organizations?
Susan Fowler: With regard to intersecting, I think that adopting microservice architecture is a natural step in the evolution of many software applications. Monorepos (monoliths) are extremely difficult to scale, and many engineering organizations can and do find themselves at a place where they reach scalability limitations (no concurrency, no partitioning, etc.). They also find that development velocity grinds to a halt, because deployment and development on a monorepo becomes incredibly difficult (from an organizational standpoint) once the application is large and there is a large number of engineers working on it. Once you get to this point, the easiest and most scalable way forward is to adopt microservice architecture.
This is precisely where most of the conflicts come from, too. Since adopting microservices is usually a "next step" in the evolution of a software system, most systems weren't designed with microservice architecture in mind, and this creates a lot of problems, some of which are organizational in nature and others technical.
For example, on the organizational side, you end up with a lot of isolated and disjointed teams who work on their own microservices but don't know anything about the rest of the system unless you plan for a lot of cross-team collaboration beforehand. You also end up with an institutionalized lack of trust between microservice development teams, who don't always know that the other microservices that their own service depends on are reliable, stable, scalable, etc.
You may also find (as many companies do) that it's difficult to staff and run a separate operations organization in microservice ecosystems, and developers will need to learn to take over operational duties for their microservices (something many of them may not be accustomed, trained, or prepared to do).
On the technical side, for example, you can end up with a lot of strange microservices that don't make much sense and don't communicate well with each other, because the boundaries between them and their functionality was never originally defined appropriately in the monolith that they were split from.
Another technical challenge I often see is that the infrastructure that underlies the microservices needs to be incredibly sophisticated for microservice architecture to be successful - something that many companies who are used to running monolithic applications aren't used to or prepared for.
InfoQ: For a developer with experience working on traditional monolithic systems, what skills are the most important to learn to successfully transition to writing microservices?
Fowler: The most important thing for developers to learn when their organization is adopting microservice architecture are the operational skills that will most likely be required of them. In monolithic applications, it's relatively easy to separate the development of the application and the maintenance of the application, and split the dev vs. ops duties between two separate teams. In microservice architecture, this isn't the case - often there will be dozens, hundreds, or thousands of microservices and, consequently, microservice development teams, and dual-staffing each of these teams with both developers and ops engineers doesn't make any organizational sense. Furthermore, microservice architecture allows devs to move so quickly that there are very few technical justifications for having operations engineers run the services - the devs will be the engineers who know it best, and they'll be able to run it best.
InfoQ: Along with new skills for developers, what cultural or organizational changes need to occur to support microservices?
Fowler: A really great question, and one that honestly requires a very long answer (which my book Production-Ready Microservices is all about!). The single most important thing that an engineering organization needs to do is to build up very stable, reliable, and sophisticated application platform infrastructure (layer three in the four-layer model of microservice architecture).
InfoQ: A successful MSA requires a significant level of infrastructure, as described in your blog post about The Four Layers of Microservice Architecture. While a single microservice is not very useful, every system has to start somewhere. How can organizations justify the overhead necessary to transition legacy systems to microservices?
Fowler: I think that most small companies wouldn't necessarily benefit from adopting microservice architecture, and I don't think that most companies should break their legacy systems into microservices.
There are several cases in which microservices work really, really well. First, if the application or system is a bit complex, but very clear boundaries can be drawn between its various functionalities, then splitting it into microservices isn't that painful of a process (provided that, as you mentioned, the required infrastructure can be put into place). I think that this is a very uncommon scenario, however.
The second scenario is the microservices story that I hear from the majority of engineers and companies: an application reaches a critical point where it is no longer scalable, its scalability limitations have introduced severe performance and stability problems, it is impossible to do any work on the application, and developer velocity has ground to a halt. In this incredibly common scenario, justifying the additional infrastructure work necessary for microservices is a no-brainer - either the infrastructure gets built, or the application won't be able to scale in the future.