Stop Over-Engineering, Build What the Customer Really Needs

After working with many different teams, Greg Young has found that they often are drastically over-engineering in their projects. Teams start to work on 9 month projects, but by thinking on the problem from another perspective they may be able to deliver 95% of the value in just a few weeks, Young claimed in his keynote at the recent DDD eXchange conference in London.

The reason for this over-engineering is that we are building the wrong thing. A huge problem for Young is that we don’t focus enough on what people actually need; we focus on what we think they want which are two very different approaches. Very often feature usage follows a Pareto distribution which means that by implementing only a few features, a high percentage of the actual usage will be covered. Continuing to cover the rest of the use cases that rarely are used will require a lot of effort but with a very little achievement.

Young emphasizes that software is just one piece of a larger system. There is an entire business process around the software and there is nothing wrong in saying that a specific issue will be solved via a business process instead of software. We talk too much about worst cases and how to solve them in software. If we automate more than 99.9% of the work, we can often leave the rest to be manually handled by domain experts.

Manual intervention required, humans come here!

Brownfield projects is one area where over-engineering tends to occur. For Young, they are also the easiest projects to prevent over-engineering, the reason being that they come with experience and data of usage. Finding the basic use cases mentioned by the domain experts and running them against actual usage will probably cover a significant part of the usage. Unfortunately, we tend to talk too much with the domain experts about edge cases that can happen and when coding for them we end up with a large amount of complexity and code that is rarely used in production. Young also notes that all this complexity really damages our models.

Greenfield projects is another area where over-engineering is common, as we don’t have access to actual usage. To overcome this, Young’s recommendation is to make an agreement with the business side to continuously deploy and then release after two months. During this time, they need to use the system, the reason being to get data in as early as possible to prevent implementing features that rarely are used. He also suggests to only work with bugs after the first release, not with features. Everything that is missing is reported as a bug. In his experience, this works very well since the business only need to prioritize one thing: bugs. He notes that this way of working is only useful for internal projects with internal users; the strategy is not suitable for fixed-price contracts or public websites.

In greenfield projects we are in imagination land

Process managers or orchestrators is the third area where Young notes that over-engineering occurs. He has seen very few projects actually succeeding in building orchestrations between multiple services. The single biggest reason is because the implementation turns into big projects flooded with accidental complexity. Even more than in the other examples, projects fail to find the main use cases. Instead, they tend to enumerate on all possible ways that things can go wrong.

Young summarizes by noting that we have to remember that our systems are taking work away from humans. Taking 99% of the work is good enough most of the time; taking the 1% left is often not economically viable.

Next year’s DDD Exchange is scheduled for late April 2017 and registration is open.