Given there’s a limit to our cognitive abilities and our comprehension of complex systems, Corstian Boerman argues that software development should become boring. He suggests moving infrastructure out of the way so that it does not burden the day-to-day development process, and focusing on delivering business value in a predictable and repeatable way.
Corstian Boerman gave a talk about how complex software impacts our cognitive abilities at NDC Oslo 2023.
Boeman mentioned that since our memory is finite and fallible, there is a limit to our comprehension of complex systems. This is important for two reasons, he explained:
First, it is expensive to train developers to memorise details. This does not only manifest itself during initial training, but also during ongoing development where low velocity is a symptom of this as well.
Additionally, the complexity boundary is reached more quickly with an irrationally complex system than it is with a natural complex system, and so I would argue we should take more inspiration from nature for the development of our systems.
There is a clear distinction between natural complex systems and human-made complex systems, Boerman said. Through physics, natural phenomena can be explained by a number of equations, while for human made systems we must take context into consideration, he added. This context -possibly irrational- can not necessarily be generalised to the same extent as is possible with natural systems.
Boerman argued that software development should become boring. He made the distinction between boring software and exciting software:
Boring software in that categorization resembles all software that has been built countless times, and will be so a billion times more. In this context, I am specifically thinking about back-end systems, though this rings true for front-end systems as well. Exciting software is all the projects that require creativity to build. Think about purpose-built algorithms, automations, AI integrations, and the like.
Making software development boring again is about laying a prime focus on delivering business value, and making the delivery of these aspects predictable and repeatable, Boerman argued. This requires moving infrastructure out of the way in such a way that it is still there, but does not burden the day-to-day development process:
While infrastructure takes most of the development time, it technically delivers the least amount of business value, which can be found in the data and the operations executed against it.
New exciting experiments may be fast-moving and unstable, while the boring core is meant to be and remain of high quality such that it can withstand outside disruptions, Boerman concluded.
InfoQ interviewed Corstian Boerman about building complex software systems.
InfoQ: How does the relationship between software complexity and cognitive abilities look?
Corstian Boerman: Cognitive ability includes three primary things: thoughts, experiences, and senses. Together we use these to understand whatever is going on around us.
Our thoughts as such can include both the conscious, slow rational system, and the unconscious, fast emotional system. Experiences can uniquely identify a human being among their peers, and we use our senses to perceive the world around us. Taken together, these define our cognitive abilities, as well as our disabilities.
As cognition itself consists of multiple aspects, extraordinary abilities and debilitating disabilities are not mutually exclusive and may as well co-exist within the same person, Boerman said, whether this is about senses (a missing limb), experiences (traumatic experiences), or our thoughts (mental illness).
Complexity and cognition are not directly related to one another. Whereas complexity is a certain characteristic of our environments, cognition is about the way we interact with our environments. Our cognitive abilities then allow us to understand complex systems, as well as to create complex systems.
InfoQ: How should we deal with boring and exciting software?
Boerman: Most of the boring systems follow the same paradigm: we have some data, and we want to run some operation against this data. When working with systems like these, we spend a lot of time getting the proper infrastructure in place. Database queries, logging statements, systems integration, and a lot more.
Contrary to boring software, there is what I consider to be exciting software as well, which is equally important to keep boring software boring. These are all the projects which require creativity to build.
In this symbiosis, the boring part contains the business processes most important to a company without any bells and whistles. This core functionality is then enriched by more exciting external software. When having these exciting bits and pieces interact with the processes defined by the boring core, it becomes possible to automate manual labour, or manually take over from automation. This provides freedom in the way a system can be evolved.