Experiences from Failing with Microservices

While being a correct philosophy regarding service design and composition, microservices can quickly lead into trouble, Richard Clayton, writes providing some observations and experiences from failing when implementing and maintaining a microservice architecture.

Richard, Chief Software Engineer at Berico Technologies, emphasizes that his team did fail while developing microservices although most of the reasons for their failure were not related to technology and implementation practices. During the project Richard found five major factors contributing to the failure.

Disagreement between developers
Different views within the team on the benefits of a microservice architecture versus a more traditional monolithic stack together with an encouraged democratic decision-making caused a lot of needless and time-consuming debate and also some emotional friction. Lack of skill among developers causing flawed implementations also led to loss of productivity.

Service boundaries causing barriers
Dividing the backend into 8 separate services followed by a decision to assign services to people enforced ownership of specific services by developers. This led to developers complaining of their service being blocked by tasks on other services and refusing to help out by working with these blocking tasks. The separation also lead to developers losing sight of the overarching system goals, instead only focusing on the service they were working on.

Independent services
Balancing the desire to share common utility code between services against independent services with replicated functionality became a huge tradeoff finally leading to a major refactoring.

Failing to specify an API for each service, particularly the model passed between each service, lead to regular issues especially for the frontend developers, a problem still not solved.

Not explicitly defining the services communication and using one mechanism for all use cases didn’t work well. Eventually they started moving towards a CQRS communication pattern, a work not yet fulfilled.

Service granularity
Starting with 8 services was a too big change for the team; probably just a couple of services would have been a better start, eventually splitting individual services into a few microservices as the need arose.

DevOps
Implementing a Continuous Delivery (CD) pipeline was probably where the team did the best, spending a lot of time learning a lot of lessons, e.g. the burden of maintaining many small services.

Richards’s recommendation for teams starting with microservices includes ensuring the team has the skills needed, and taking a pragmatic approach, adapting to teams and customers capabilities.