BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Simplicity, Please - A Manifesto for Software Development

Simplicity, Please - A Manifesto for Software Development

This item in japanese

Bookmarks

Key Takeaways

  • Complexity is the greatest impediment to a software company’s growth and profitability.
  • Development costs increase exponentially with respect to the complexity of your code base.
  • The choice between speed and simplicity is a false dichotomy. Investment in simplicity is investment in speed.
  • Few executives recognize the cost of complexity because they don’t understand the nature of the products they’re selling.
  • Software is a product like no other — a hybrid product-service with a very unique warranty.

Complexity is a characteristic of your company’s source code. It’s usually the consequence of several failings. Recognize any of these?

  • An ill-conceived architecture i.e. the manner in which data passes between application components
  • An abundance of dead code (code that’s no longer in use)
  • Code expressed without regard to established patterns like object-oriented design and dependency injection
  • Badly maintained structures for storing data such as unused columns or decommissioned tables in a relational database
  • Inadequate in-line documentation, making the code difficult to read by developers who must maintain or extend its functionality

Software developers have a name for those kinds of failings. We call them technical debt. No one likes to talk about it, but the cumulative cost of every “could’ve,” “would’ve,” and “should’ve” in the evolution of your company’s code will imperceptibly take a stranglehold on product development. Features that might have taken days or weeks to roll out when the company was younger now require months of development time.

Simplicity is the counterpart of complexity. Code exhibits simplicity when it’s well organized, logically minimal (or close thereto), and easily readable. Just to be clear, complexity and simplicity bear no intrinsic relation to the size of the code base. A large code base (Google's is some 2 billion lines) need not be complex, and a small code base may not exhibit simplicity.

The Complexity Problem

Complexity is one of the greatest impediments to a software company’s growth and profitability. Development costs increase exponentially with respect to complexity. This exponential relationship exists because, as a software product becomes more complex, more developers are necessary to extend and maintain it. And as the number of developers increases, the company's managers are less able to prevent additional increases in complexity.

So why do so many software companies ─ most of which are blessed with smart and industrious people whose livelihoods are vested in the success of their software ─ allow complexity to imperil the code base? Few executives (and even fewer angel investors and venture capitalists) recognize the cost of complexity because they don't understand the nature of the products they're selling. Software is a product like no other. When you sell software, you're actually selling a hybrid product-service with a very unique warranty (see inset A or B? below).

The executives of software companies, even the erstwhile engineers, are predominantly graduates of business school programs. They view their industry from the perspectives of sales, marketing, and microeconomics, and they are propelled by the dogma of first-mover ─ or, more generally, fast-mover ─ advantage.

It's good to be first, but it's better to be the best. Early search pioneer Yahoo! discovered this when Google entered the search market much later with a superior algorithm, business model, and leadership team. Of course, at some point, a company can be just too late to the party. It would be difficult to conceive, for example, of challenging Google's leadership position in search nowadays. But the race to industry dominance is a marathon, not a sprint, and those first few laps around the track have very little bearing on the longer-term outcome.

By demanding that middle managers bring products to market as quickly as possible, a monomaniacal focus on fast-mover advantage can imperil the company to the point of extinction. Meanwhile, workflow management methodologies such as Agile ─ so often misunderstood and badly implemented ─ may compound the problem by institutionalizing practices that place a premium on speed. Simplicity is the silent victim. Software developers are often the only ones in the company to realize this.

Of course, a software company can't generate revenue from a product still undergoing design or assembly. But investment capital and return on capital employed are at greatest risk when executives choose the speed of release over the integrity of the code base. The cost of complexity assuredly weighs heavier than any savings from an accelerated release date.

Ironically, the choice between speed and simplicity is a false dichotomy. Investment in simplicity is an investment in speed, for simplicity provides the means to bring software products to market more quickly, and then maintain and enhance them more efficiently. It’s also the mother lode of intellectual property ─ and a competitive advantage that’s almost impossible to regain once lost.

Zeno's Paradox (and Yours Too?)

In Achilles and the Tortoise ─ one of the paradoxes of Ancient Greek philosophy — a fleet-of-foot Achilles must overtake a ponderous tortoise favored with an arbitrary head-start. Zeno posits that because the pursuer must always first catch up to the point from which the pursued began, and because Achilles must do so for every increment of distance subsequently traversed by the tortoise, the tortoise will always maintain a lead ─ albeit an ever-diminishing lead.

Zeno’s paradox is helpful in explaining why your company may also be an Achilles chasing a tortoise. When source code becomes complex, it’s harder to tack on new code, and old code can become almost impossible to debug. Developers, who have an intrinsic bias towards simplicity and order, become frustrated. Then they quit. Of course, not all developers quit, and those who do likely won't all quit at once. Nevertheless, the pervasive impact is material, and your company's rate of progress diminishes with their departure and the associated costs of recruitment and onboarding.

Moreover, you'll have difficulty hiring replacements because, in today's socially networked world, the state of your code is essentially public knowledge among the community of developers from which you’ll need to recruit. To the extent you can hire at all, the compensation paid to your engineers to work with a complex code will need to be higher-than-market and trending ever upward. Meanwhile, the addition of features and the ensuing escalation in complexity will dictate the need for yet more engineers. So the death spiral continues.

The software company in which I work has confronted the complexity problem. Like many others, I don't think we clearly recognized the causes or effects of our predicament. And like many others, we turned to outsource (both onshore and offshore) in hope of ameliorating the situation. But complexity is a problem that can't be outsourced away. Outsourcing only makes the problem worse because the outsourcing entities, and the contractors whom they employ, have an economic incentive to horde knowledge and increase the complexity of the code.

A or B?

Imagine this. Competition among auto manufacturers and a glut of used car inventory are suppressing new car sales. So the executives of one luxury manufacturer respond with a bold move: the Forever New warranty. For a 20% hike on the sticker price ─ a premium derived from careful price-elasticity-of-demand analysis ─ they'll offer customers not only the same high-quality automobile, but also the promise to take complete responsibility for the vehicle throughout the warranty period (say, four or five years), including all scheduled maintenance plus retrofitting new features as they become available on later models.

Let's assess the merits of this initiative under two scenarios.

Scenario A: The company's vehicles are well designed. Its engineers have exploited the benefits of modularity and an open architecture, so it's relatively easy to add features or replace malfunctioning components. Not only does new vehicle revenue increase by 20%, but also the sales and service departments of the company's dealerships are busier than ever. Meanwhile, environmentalists, government agencies, and drivers laud the company for maintaining vehicles with continually improved efficiency and safety features. Owners also delight in the fact that the resale value of their vehicles at the end of the warranty period is considerably higher, mitigating the premium paid for Forever New. The innovation is soon heralded as a triumph. Within a couple of years, other manufacturers are experimenting with “forever new” warranties of their own, but inferior design of their vehicles – that is, complexity – renders such experiments tentative.

Scenario B: Although the company's vehicles are feature rich and perform well, under the hood it's all a bit of a mess. The 20% increase in new vehicle sales revenue since the introduction of the program goes some way to offsetting the costs of ongoing maintenance and feature enhancement. But with each successive enhancement, the difficulties and costs of honoring the warranty escalate. Barely two years into Forever New, those costs are unsustainable. The auto manufacturer withdraws the offer. It must still, of course, honor its obligation to existing customers. Executives estimate losses at almost one billion dollars and face a difficult decision about whether to continue operations.

Now, you can point out the impracticalities of Forever New in the automobile industry. But if you’re an executive at a software company, you should recognize that you’re already offering a Forever New warranty. The only question is, Are you operating under scenario A or B?

A Manifesto for Common Sense

This is a manifesto for better software development, or more modestly, a plea for common sense. If your organization isn't yet past the breaking point, follow the prescription below. It's a road map to unwind the complexity of your software, achieving along the way markedly lower development costs, faster time to market, and a sustainable development process that secures the future of your company while leaving competitors in your wake.

Learn. Complexity can only be fixed from the top down. Leaders need to invest time to understand everything that happens along the software development production line. How much time? You'll know you've accomplished your goal when time-to-market is a consideration secondary to the simplicity of your code.

Lead. Instill an emphasis on simplicity through the ranks of engineering management so your organization is aligned around a new set of values. Communicate the strategic value of this initiative.

Reevaluate. Assess whether workflow management methodologies such as Agile really better fit your needs than alternative approaches that emphasize design quality over implementation speed.

Hire strategically. Recruit developers who love simplicity. These are people who'll typically be among the slowest to write actual code, favoring instead lengthy periods of time to analyze requirements and architect optimal solutions in terms of the code’s maintainability, execution time, memory consumption, and data storage. They’ll also take pride in documenting their work and sharing knowledge with peers.

Manage knowledge. Knowledge of your code is the lifeblood of your company, and that knowledge needs to diffuse throughout the organization rather than reside in localized silos. Maintaining a diffuse knowledge structure means investing more time in knowledge sharing, but will allow you to quickly redeploy engineers according to the changing needs of the business.

Keep your eyes on the road. A beguiling and ever-growing array of technologies and tools now exists for writing, testing, deploying, and monitoring software. Most are largely duplicative, conferring only incremental benefits and high switching costs. Make smart choices on a small subset of these offerings. Give primary consideration to the technologies your engineers already know.

Measuring Your Progress

You can benchmark your company's progress in the war on complexity by tracking the number of engineers you employ and their rate of turnover. If you're outside the high-tech industry and the size of your engineering department is growing faster than your online sales, that's a really bad sign. If you're a software company hemorrhaging staff, that's bad too. A company whose source code is simple will have a relatively small and stable team of engineers, and that team will grow slowly even as the functionality of its products marches forward. Conversely, a company whose software is plagued by complexity will find itself continuously hiring, running ever faster in a breathless effort to chase down a tortoise.

About the Author

Paul Merlyn is a full-stack engineer and thought leader at Consensus Corp., the digital activation subsidiary of Target. He’s also an active entrepreneur and founder of Fynancially (a venture to democratize the financial services industry), the Abridg social network, and the National Mediation Training Registry. Simultaneously opinionated and diplomatic, he believes JavaScript is eating Java’s lunch and will unequivocally rule the world by dinner time.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Perls of Wisdom

    by Edwin Dalorzo,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I had been looking for words to express how I feel too and your manifesto taught me the way to do so. I couldn't agree more with all this, complexity is our silent killer as well as a mistaken agile mentality focused only on the speed of delivery. It is clear in the trenches that we need to refocus our vision, but it is also difficult to influence the organization from the bottom up. What would be your suggestion for a better software methodology that fosters this simplicity? What would be your advice to influence a change of mentality from the bottom up?

  • Simplicity, Please - A Manifesto for Software Development

    by Bertrand Gajac,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I fully agree with the fact that simplicity is very important, and that is true not only for software development. In everyday's life we have problems that may be simple or complex. One must always find the simplest solution. When the problem is complex, the solution can be complex. When the problem is simple,the solution must be simple. In all cases, the complexity of the solution must not be greater than the complexity of the problem. People who find complex solutions for simple problems usually don't find any solution for complex problems.

  • Modularity

    by tctec ptyltd,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Perhaps it is Modularity you need. Because the 'Simplicity' you describe is the elegant solution only visible in hindsight.

  • Developers resposibility

    by Javier Paniza,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    You said that the managers are the responsible, but I think that developers are responsible too.

    In order to achieve simplicity, many times you have to renounce to use a nosql db, or the latest JS framework, or microservices, or the last hot language, or... Most times "the simpler approach to solve a problem" is boring for developers.

    For developers, and for software architects, the best way to achieve simplicity is always to rewrite the software from scratch, so you have a simpler software (in two years) that achieve the 90% of the needed features, ton you achieve the 100% of needed feature you need 3 years more, and the software will be complex again, and obsolete again.

    The developers should think in the simpler way to achieve the customer satisfaction (not the developer satisfaction). I think that the only way to achieve this it's making the developer salary dependent on revenues, so they worry a little more about the impact of their actions in the customers.

  • How do you define simplicity vs complexity?

    by John Broderick,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    I'm guessing that you're talking about simplicity in the code base but without a definition it's difficult to determine.
    How do you define simplicity vs complexity? Employee turnover can be an indicator of a complex code base, but it's can also be caused by other reasons so it's not a reliable indicator.

    Rich hickey gives a good overview of simple vs complex in this talk www.infoq.com/presentations/Simple-Made-Easy

  • Re: Perls of Wisdom

    by Paul Merlyn,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thank you for your comment, Edwin. I'm really arguing with myself in suggesting ways to promote simplicity from the bottom up because I believe, as my article asserts, "Complexity can only be fixed from the top down." That said, the spirit of your question is well taken. Anyone in an organization should be able to promote change, no matter where they reside on the org chart. To that end, I'd suggest engineers, for instance, practice a little organizational jiu-jitsu, using the language of Agile to equate agility with simplicity. That's probably the more effective tactic lower down in the organization. At the C-level, however, I'd suggest calling more attention to the economic cost of complexity and the associated ROI of driving complexity out of the organization.

  • Re: Modularity

    by Paul Merlyn,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks for your comment. It's difficult to identify the details of simplicity in the abstract. I think we always need the full context of a specific case. However, whatever the case, the principles of simplicity need to apply across enterprise's production line, from architecture/system design all the way through to implementation. That being said, sometimes modularity might be the appropriate pattern, other times monolithic -- but always as simple as possible.

  • Re: How do you define simplicity vs complexity?

    by Paul Merlyn,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Thanks for your comment, John, and for sharing that link to Rich Hickey's talk! Rich offers a much more in-depth and explicit definition of simplicity than I've attempted. I'll just suggest that simplicity is best measured by the ease with which a product design (e.g. a software application) can pass through the organization into the hands of its users and then remain vital to that user base as needs and technologies evolve.

    To make myself clearer: I'm comparing simplicity with the property of aerodynamism. A car or an airplane needs to convey its passengers through the air efficiently. Achievement of an aerodynamic design is likely an iterative process of obsessive refactoring. I'm not the first to observe that aerodynamic designs are always aesthetic. That correlation, I believe, can similarly help us recognize that software exhibits simplicity when its design and implementation are beautiful.

  • Re: Developers resposibility

    by Paul Merlyn,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    You raise an excellent point, Javier. I agree that the pull for developers and architects to experiment with new technologies can impede the quest for simplicity. This is why executives need to understand simplicity and complexity much more deeply than is common at present. Without that understanding, they won't be able to keep the perils of complexity at bay.

    Of course, I'm not a luddite towards new technologies. Nor would I agree that a rewrite from scratch is always the best way to promote simplicity in software. New technologies are often a more promising avenue to introducing greater simplicity e.g. a GraphQL server can radically simplify a microservices sequence diagram. However, sometimes that shiny new technology really isn't facilitative of anything.

  • Measuring Progress

    by Yvonne McGinnis,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Love your standards and criteria for measuring progress within a business enterprise. My sentiments exactly. Very well stated! Especially loved the statement “Conversely, a company whose software is plagued by complexity will find itself continuously hiring, running ever faster in a breathless effort to chase down a tortoise.”

  • Re: Measuring Progress

    by Paul Merlyn,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Many thanks for your kind feedback, Yvonne!

  • Re: Measuring Progress

    by Yvonne McGinnis,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Your welcome Paul; you credit where it’s due (at least I do).

  • Simplicity using Artificial Intelligence

    by Chris Hillman,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Paul Merlyn in “Simplicity Please” really does spell out the major problems facing the IT industry and most of those are not being addressed directly. We spend time trying to ameliorate the problems by clever ways of reducing the symptoms, or the consequences.

    Let me introduce you to ENIGMA: an Artificial Intelligence, Automated System Development platform. I think ENIGMA addresses the cause of the problems: but I admit I am biased. I wrote the platform.

    Using AI means that the program structure will be the same throughout an application. It will be a simple structure. AI also means that each program will do one task; effectively a “black box”. ENIGMA applies AI by generating 99% of the application code from the products of Analysis and Design.

    ENIGMA uses a patented application structure where the User interacts with programs running with an Internet platform; either remotely or locally. In turn, these programs interact with a single Database maintenance and interrogation program; the so called, Core program. This structure has a unique effect on the application: the User interacts with Information and the Core interacts with Data. The SQL required by the Core program is 100% generated by AI.

    There is a lot more I could say. Instead, I invite you to visit the web site at Parametric Systems, where much more documentation is available.

    Come and browse the future. Its real, it is now and it works.

    ENIGMA! It’s a Revolution, not an Evolution.

  • Sorry

    by Yvonne McGinnis,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Paul that statement should read, you give credit where it’s due (at least I do).

  • Simplicity might not be what you think it is

    by Florin Jurcovici,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Most complexity I see nowadays comes from complexity in the problem domain. More often than not, that's because of dysfunctional organizations. The software you build will reflect the organization's communication patterns - Conway's law. You can't get around it. But, if you are a solid professional, you can help your customers recognize the problems they have in their processes. More often than not, if you get them to that point, you'll be able to sort out things and come up with simpler, better processes.

    Another huge source of complexity is what I call CV-driven development - pull in all that's available on maven central (or on npm or on pypi, it's all the same), so you can then add everything to your CV. Developers no longer develop the mental model of a working computer, to understand what pulling in a library does to their project. They end up with binaries several GB in size, even for tiny problems, and don't understand why this is wrong, or what consequences it can have.

    Another important source is orthodoxy. Most young, ambitious Java developers I encounter define one interface for each class they implement, even if it's obvious there will only ever be a single implementation of that interface. Even if developers should know, nowadays, that documenting code is an anti-pattern, many organizations _require_ developers to write code comments. (That's OK for closed source for which you generate documentation from code, provided the documentation goes through a testing process similar to that of the code itself - but that almost never happens. For open source, you need provide generated documentation only if your code isn't readable - howtos or user guides can not be generated from code.) The anemic domain model is indeed an anti-pattern, when you actually have a domain model, but it's the sanest implementation for purely CRUD apps, yet many orgs don't make a distinction. Some frameworks/products/whatever recommend creating facades even when they're a one to one match for what they are supposed to hide complexity for.

    What you listed is to a large extent just lack of professionalism - aside from inline documentation. An excess of inline documentation is a code smell - insufficiently expressive code. If you have this problem, spend more time on making your code expressive. If you aren't able to write readable code, what makes you think you'll be able to write readable prose about the unreadable code? Plus, there's a more subtle problem with inline documentation. Except for rare situations where you absolutely have to document awkward solutions caused by external forces (workarounds for bugs, oddities in called APIs or similar), inline documentation says your architecture is of such nature that it can't be implemented in an expressive, self-describing way. _That_ is what you should fix. Documenting it is just putting lipstick on a pig.

    The sources I've listed above are much meaner. You can clean up dead code - every decent IDE helps you with that. Refactoring to patterns - provided you know what you're doing - is well documented and again, there's strong IDE support. In a professionally maintained product, database structure should be versioned similarly to code, and this should take care of dead data. However, it an anemic domain model doesn't pass code review, and coding standards force you to implement a facade for it, and then document plain simple accessors and mutators, and you pull in a huge object mapper just so you can easily convert a flat map of properties into many different representations, you have just created a whale of an application out of something that could have been an extreme lightweight one. It will be very beneficial for you, because you'll need to charge the customer a ton of money to maintain it, but that way of working is definitely not ethical. _This_ is the kind of complexity that's damaging. Few people nowadays are able to recognize it, IME. Most people in control of money at the customer don't understand a thing about programming, so people buy such BS. That's the bane of our industry.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT