Key Takeaways
- Complexity is the greatest impediment to a software company’s growth and profitability.
- Development costs increase exponentially with respect to the complexity of your code base.
- The choice between speed and simplicity is a false dichotomy. Investment in simplicity is investment in speed.
- Few executives recognize the cost of complexity because they don’t understand the nature of the products they’re selling.
- Software is a product like no other — a hybrid product-service with a very unique warranty.
Complexity is a characteristic of your company’s source code. It’s usually the consequence of several failings. Recognize any of these?
- An ill-conceived architecture i.e. the manner in which data passes between application components
- An abundance of dead code (code that’s no longer in use)
- Code expressed without regard to established patterns like object-oriented design and dependency injection
- Badly maintained structures for storing data such as unused columns or decommissioned tables in a relational database
- Inadequate in-line documentation, making the code difficult to read by developers who must maintain or extend its functionality
Software developers have a name for those kinds of failings. We call them technical debt. No one likes to talk about it, but the cumulative cost of every “could’ve,” “would’ve,” and “should’ve” in the evolution of your company’s code will imperceptibly take a stranglehold on product development. Features that might have taken days or weeks to roll out when the company was younger now require months of development time.
Simplicity is the counterpart of complexity. Code exhibits simplicity when it’s well organized, logically minimal (or close thereto), and easily readable. Just to be clear, complexity and simplicity bear no intrinsic relation to the size of the code base. A large code base (Google's is some 2 billion lines) need not be complex, and a small code base may not exhibit simplicity.
The Complexity Problem
Complexity is one of the greatest impediments to a software company’s growth and profitability. Development costs increase exponentially with respect to complexity. This exponential relationship exists because, as a software product becomes more complex, more developers are necessary to extend and maintain it. And as the number of developers increases, the company's managers are less able to prevent additional increases in complexity.
So why do so many software companies ─ most of which are blessed with smart and industrious people whose livelihoods are vested in the success of their software ─ allow complexity to imperil the code base? Few executives (and even fewer angel investors and venture capitalists) recognize the cost of complexity because they don't understand the nature of the products they're selling. Software is a product like no other. When you sell software, you're actually selling a hybrid product-service with a very unique warranty (see inset A or B? below).
The executives of software companies, even the erstwhile engineers, are predominantly graduates of business school programs. They view their industry from the perspectives of sales, marketing, and microeconomics, and they are propelled by the dogma of first-mover ─ or, more generally, fast-mover ─ advantage.
It's good to be first, but it's better to be the best. Early search pioneer Yahoo! discovered this when Google entered the search market much later with a superior algorithm, business model, and leadership team. Of course, at some point, a company can be just too late to the party. It would be difficult to conceive, for example, of challenging Google's leadership position in search nowadays. But the race to industry dominance is a marathon, not a sprint, and those first few laps around the track have very little bearing on the longer-term outcome.
By demanding that middle managers bring products to market as quickly as possible, a monomaniacal focus on fast-mover advantage can imperil the company to the point of extinction. Meanwhile, workflow management methodologies such as Agile ─ so often misunderstood and badly implemented ─ may compound the problem by institutionalizing practices that place a premium on speed. Simplicity is the silent victim. Software developers are often the only ones in the company to realize this.
Of course, a software company can't generate revenue from a product still undergoing design or assembly. But investment capital and return on capital employed are at greatest risk when executives choose the speed of release over the integrity of the code base. The cost of complexity assuredly weighs heavier than any savings from an accelerated release date.
Ironically, the choice between speed and simplicity is a false dichotomy. Investment in simplicity is an investment in speed, for simplicity provides the means to bring software products to market more quickly, and then maintain and enhance them more efficiently. It’s also the mother lode of intellectual property ─ and a competitive advantage that’s almost impossible to regain once lost.
Zeno's Paradox (and Yours Too?)
In Achilles and the Tortoise ─ one of the paradoxes of Ancient Greek philosophy — a fleet-of-foot Achilles must overtake a ponderous tortoise favored with an arbitrary head-start. Zeno posits that because the pursuer must always first catch up to the point from which the pursued began, and because Achilles must do so for every increment of distance subsequently traversed by the tortoise, the tortoise will always maintain a lead ─ albeit an ever-diminishing lead.
Zeno’s paradox is helpful in explaining why your company may also be an Achilles chasing a tortoise. When source code becomes complex, it’s harder to tack on new code, and old code can become almost impossible to debug. Developers, who have an intrinsic bias towards simplicity and order, become frustrated. Then they quit. Of course, not all developers quit, and those who do likely won't all quit at once. Nevertheless, the pervasive impact is material, and your company's rate of progress diminishes with their departure and the associated costs of recruitment and onboarding.
Moreover, you'll have difficulty hiring replacements because, in today's socially networked world, the state of your code is essentially public knowledge among the community of developers from which you’ll need to recruit. To the extent you can hire at all, the compensation paid to your engineers to work with a complex code will need to be higher-than-market and trending ever upward. Meanwhile, the addition of features and the ensuing escalation in complexity will dictate the need for yet more engineers. So the death spiral continues.
The software company in which I work has confronted the complexity problem. Like many others, I don't think we clearly recognized the causes or effects of our predicament. And like many others, we turned to outsource (both onshore and offshore) in hope of ameliorating the situation. But complexity is a problem that can't be outsourced away. Outsourcing only makes the problem worse because the outsourcing entities, and the contractors whom they employ, have an economic incentive to horde knowledge and increase the complexity of the code.
A or B?
Imagine this. Competition among auto manufacturers and a glut of used car inventory are suppressing new car sales. So the executives of one luxury manufacturer respond with a bold move: the Forever New warranty. For a 20% hike on the sticker price ─ a premium derived from careful price-elasticity-of-demand analysis ─ they'll offer customers not only the same high-quality automobile, but also the promise to take complete responsibility for the vehicle throughout the warranty period (say, four or five years), including all scheduled maintenance plus retrofitting new features as they become available on later models.
Let's assess the merits of this initiative under two scenarios.
Scenario A: The company's vehicles are well designed. Its engineers have exploited the benefits of modularity and an open architecture, so it's relatively easy to add features or replace malfunctioning components. Not only does new vehicle revenue increase by 20%, but also the sales and service departments of the company's dealerships are busier than ever. Meanwhile, environmentalists, government agencies, and drivers laud the company for maintaining vehicles with continually improved efficiency and safety features. Owners also delight in the fact that the resale value of their vehicles at the end of the warranty period is considerably higher, mitigating the premium paid for Forever New. The innovation is soon heralded as a triumph. Within a couple of years, other manufacturers are experimenting with “forever new” warranties of their own, but inferior design of their vehicles – that is, complexity – renders such experiments tentative.
Scenario B: Although the company's vehicles are feature rich and perform well, under the hood it's all a bit of a mess. The 20% increase in new vehicle sales revenue since the introduction of the program goes some way to offsetting the costs of ongoing maintenance and feature enhancement. But with each successive enhancement, the difficulties and costs of honoring the warranty escalate. Barely two years into Forever New, those costs are unsustainable. The auto manufacturer withdraws the offer. It must still, of course, honor its obligation to existing customers. Executives estimate losses at almost one billion dollars and face a difficult decision about whether to continue operations.
Now, you can point out the impracticalities of Forever New in the automobile industry. But if you’re an executive at a software company, you should recognize that you’re already offering a Forever New warranty. The only question is, Are you operating under scenario A or B?
A Manifesto for Common Sense
This is a manifesto for better software development, or more modestly, a plea for common sense. If your organization isn't yet past the breaking point, follow the prescription below. It's a road map to unwind the complexity of your software, achieving along the way markedly lower development costs, faster time to market, and a sustainable development process that secures the future of your company while leaving competitors in your wake.
Learn. Complexity can only be fixed from the top down. Leaders need to invest time to understand everything that happens along the software development production line. How much time? You'll know you've accomplished your goal when time-to-market is a consideration secondary to the simplicity of your code.
Lead. Instill an emphasis on simplicity through the ranks of engineering management so your organization is aligned around a new set of values. Communicate the strategic value of this initiative.
Reevaluate. Assess whether workflow management methodologies such as Agile really better fit your needs than alternative approaches that emphasize design quality over implementation speed.
Hire strategically. Recruit developers who love simplicity. These are people who'll typically be among the slowest to write actual code, favoring instead lengthy periods of time to analyze requirements and architect optimal solutions in terms of the code’s maintainability, execution time, memory consumption, and data storage. They’ll also take pride in documenting their work and sharing knowledge with peers.
Manage knowledge. Knowledge of your code is the lifeblood of your company, and that knowledge needs to diffuse throughout the organization rather than reside in localized silos. Maintaining a diffuse knowledge structure means investing more time in knowledge sharing, but will allow you to quickly redeploy engineers according to the changing needs of the business.
Keep your eyes on the road. A beguiling and ever-growing array of technologies and tools now exists for writing, testing, deploying, and monitoring software. Most are largely duplicative, conferring only incremental benefits and high switching costs. Make smart choices on a small subset of these offerings. Give primary consideration to the technologies your engineers already know.
Measuring Your Progress
You can benchmark your company's progress in the war on complexity by tracking the number of engineers you employ and their rate of turnover. If you're outside the high-tech industry and the size of your engineering department is growing faster than your online sales, that's a really bad sign. If you're a software company hemorrhaging staff, that's bad too. A company whose source code is simple will have a relatively small and stable team of engineers, and that team will grow slowly even as the functionality of its products marches forward. Conversely, a company whose software is plagued by complexity will find itself continuously hiring, running ever faster in a breathless effort to chase down a tortoise.
About the Author
Paul Merlyn is a full-stack engineer and thought leader at Consensus Corp., the digital activation subsidiary of Target. He’s also an active entrepreneur and founder of Fynancially (a venture to democratize the financial services industry), the Abridg social network, and the National Mediation Training Registry. Simultaneously opinionated and diplomatic, he believes JavaScript is eating Java’s lunch and will unequivocally rule the world by dinner time.