Fred George talked about the Challenges in Implementing MicroServices and The Secret Assumption of Agile at the GOTO Amsterdam 2015 conference. InfoQ interviewed him about how make microservices as small as possible, challenges when implementing microservices and how to deal with them, why programming style matters, and what developers can do to develop their code writing skills.
InfoQ: Can you give your view on how small microservices should be?
George: There are lots of answers relative to size. I subscribe to the sentiment expressed in Berlin at the MicroXchg conference: A single programmer can design, implement, deploy and maintain a microservice. I do go a bit further, borrowing on a principle from Object-Oriented Programming: A service is as small as possible, yet still does something interesting. And by "something interesting", I don’t mean just throw your data at anyone who asks. Rather, a service should be drawing conclusions and publishing that information to subscribers.
In practice, my microservices are less than 100 lines of executable code, and usually less than 20. And they still do something interesting.
InfoQ: What can you do to get them so small?
George: There are some signs I watch out for when I develop my services. If I think about the service and use the word "and", I can probably break them apart. For example in the Need pattern, one service expresses the need on a message bus, and another service collects the possible solutions. There is no need to code this as a single microservice; these can be two, different "jobs".
Second, if I am feeling a need to use Threads, I definitely think I can split it apart.
InfoQ: What are the main challenges that you see when companies are implementing microservices?
George: The biggest stumbling block for microservice adoption will be the organization trying to shoehorn microservices into their existing IT processes. Many IT organizations are structured around building larger services or even monolithic applications. Processes necessary for these architectures don’t port well to microservices.
This barrier also extends to the business side. I don’t believe in Conway’s Law that the software mirrors the business structure; I have broken it too many times. However, I do believe that many organizations have been structured that way because that is how projects are funded.
So both the IT and the business need to be re-tuned to be productive doing microservices. IT needs to manage around business KPI’s, both monitoring them and building microservices to improve them. The business must shift from throwing requirements over the wall, to working closely with the IT group trying ideas to improve the KPI’s.
In summary, microservices alone will not improve the company. It needs corresponding shifts in deployment (Ops), business interactions, and organization of the IT staff itself.
InfoQ: Can you give some advice how to deal with those challenges?
George: First, we need to assess whether microservices is a viable solution for the challenges a company faces. Microservices are particularly well-suited for certain classes of problems, particularly the "fuzzy" problems like recommendation engines or acceptable risk. A traditional accounting application, for example, accrues less benefit from microservices.
Second, we need to reset expectations at the highest levels of the company, particularly since the effective use of microservices has broader impact.
Finally, microservices should be piloted with a specific, important problem being solved. The pilot needs to be free from traditional organizational requirements around process, roles, approvals, etc. Learnings from the pilot will guide broader adoption, and hint at the fundamental changes needed across the organization for success.
InfoQ: Can you elaborate why you think the programming style of developers on the team matters?
George: Much of the software engineering practices of Agile originate from the XP process Ken Beck documented in the late 90’s. Kent and his colleagues were Smalltalk programmers, and used a particular style of programming that facilitated easy change. That allowed the team to play a new story without having all the requirements up front, along with a corresponding architecture and design. It is no accident that the refactoring editors were developed in Smalltalk originally.
Fortunately, this style of Smalltalk programming ports well to Java, C#, Visual Basic, Python, Ruby and the like.
So exploiting this style of programming readily supports the Agile assumption that you can play stories in any order, and that a lot of up-front design is unnecessary.
IT shops that feel up-front design is essential are not using the Smalltalk style. Indeed, they are unlikely to be even aware that such a style even exists. I talk about this in my presentation, The Secret Assumption of Agile. When I engage to develop code for a company, I first train the team in these techniques before we start developing anything!
InfoQ: What can happen if the programming style is not well aligned with Agile?
George: All new, greenfield programming projects start fast. The style difference doesn’t kick in until later. If the team uses techniques they learned in school (and I was taught in the early 70’s), it gets harder and harder to change the system. Architecture and design shortcomings are blamed, and eventually a rewrite is necessary. History repeats itself.
A major consulting firm reported that Agile improves software development 13% at best. I typically get 4x or better improvement. I do believe the difference is in style.
Remember the GOTO statement? It’s use was deemed in the late 70’s as the source of many errors, and needs to be abandoned. About 12% of all the statements I was writing were GOTO statements! How can I write a program without it? Today, the statement doesn’t exist in most languages. In this Smalltalk style programming, we almost never use ELSE statements. Open up a typical piece of code, and count the ELSE statements. Much like GOTO, we need to change our style to eliminate it.
InfoQ: Can you give some examples of successful coding styles? What makes them successful?
George: Client confidentiality prevents me from citing too many examples from projects I worked on. One of my early Agile projects was with Nationwide Insurance in the US, and was the subject of a white paper by Forrester Research about Agile effectiveness. The study cites that using Agile practices, we produced the product at 1/6th the effort of a top, CMM Level 5 offshore firm. The study further estimates the ROI of the project at 94% versus less than 4% for the off-shore solution.
The application was a rewrite of a legacy system. The before and after technologies were the same: Java, Oracle, and a Web front end. The legacy system had 72 Java classes. The rewrite had 1400. Since then, the new system has had a 10x higher quality than any other software they have developed.
InfoQ: What can developers do to develop their code writing skills?
George: If you pursue programming as your career, then like doctors, you need to continually invest in learning. And just as you wouldn’t want to have surgery by a doctor who has only studied the procedure, you don’t want code from a programmer who has only studied a new language or technology. The skills are developed by practicing them, and by studying under someone who has mastered the skill.
For me, I enjoy pair programming as the efficient learning vehicle. By working with someone who knows the technology better, I can get better. And in return, my pairing partner picks up some skills that I have. At the end of the day, you have two, slightly better programmers. Do this for several months, and the skills growth is phenomenal.
If your work is boring, and the tech is even more boring, find yourself a Meet-Up or an open source project, and then work with someone there. Don’t neglect to invest in your most important asset: Yourself!