*-Driven* do not change anything
Have you ever heard that a software professional needs to master a *Driven* approach to be Software Professional? These might be: Domain-Driven Design, Test-Driven Development, Behavior-Driven Development, Data-Driven Design, Data-Driven Development, Use Case-Driven Design, Use Case-Driven Development, Architecture-Driven Design, Architecture-Driven Development, Model-Driven Development, Agile Model-Driven Development and so on, and so forth...I have.
I think we expect to much magic from these *Driven* approaches (or mental frameworks as I call them) because we simply like frameworks. We often forget that they play a supporting role to our skills and are not the only goal for a software professional. But this love of frameworks exposes us to some cognitive biases which I would like to give you an insight into.
How mental frameworks are created?
So-called *Driven* approaches are examples of a class of tools that can be called mental frameworks. A mental framework, just as in the case of a software framework, is a construct (a set of ideas or procedures) that can be used to perform development activities such as modeling, designing, programming or testing in an orderly manner. We will take a look at three examples of mental frameworks: Domain-Driven Design, Test-Driven Development, Behaviour-Driven Development. These are quite popular on the Community.
The assumption of people who create these frameworks is that they will facilitate the work of developers through:
- making some general ideas (for example object modeling or testing) more concrete,
- introducing an orderly manner of work,
- presenting complex activities in the form of repeatable steps.
It all boils down to a promise and a catchy meme
When an author presents a mental framework, it means a promise that from now on everything will be better, more effective, faster and cheaper – that it will all be great. Usually, this promise refers to problems encountered by developers in the process of applying a particular concept, and it arbitrarily assumes that these problems will be solved when the suggested framework is used. Test-Driven Development is a good example of this. Some of its promises include: “you won’t have to use a debugger”, “you’ll have a simple and quality code”, “your code will have less bugs when it’s in production”. It would be totally unfair if you had an impression that I claim that these are empty promises. This is not the case. What I mean is that these promises were not supported by unambiguous and verifiable data in early years of TDD. The enthusiastic reaction to TDD came first and then some measurements were made to verify its promises.
So what makes those mental framework promises so “catchy”? Definitely, one of the factors contributing to this situation is our hope of having all our problems solved. Hence, the bigger the problems, the bigger the chances that we will catch it. Another factor is the contagious meme. A meme is a unit of cultural information in the same way that a gene is unit of genetic information, and in the same way as a byte is unit of digital information. The meme term was coined by Richard Dawkins – British evolutionary biologist and writer – for an idea considered as a replicator.
Each framework defines its meme – contagious slogan, which makes developers associate it with a particular framework and its promises. It often happens that this meme refers to one of the main concepts promoted by this framework. A nice example of it is a TDD meme, i.e. Red-Green-Refactor, which is nothing different than an exact implementation of Plan-Do-Check-Act cycle (PCDA or the Deming cycle, in short), which is an iterative and adaptive process for the continuous improvement of products and services. Red-Green-Refactor, this short slogan has the following features:
- it presents the basic TDD technique in three words,
- it is easy to remember,
- it refers to the visual tool element supporting TDD.
These three words enable you to describe TDD in a short text message. You do not have to present an elaborate concept, define it in detail and distinguish between specific cases or complicated procedures. It is as simple as Red-Green-Refactor.
In Table 1 I present examples of mental frameworks, their promises and their catchy memes.
Framework It promises you that Catchy memes TDD you will eliminate bugs visible in production almost completely and generate no more code than required Red-Green-Refactor, Unit testing BDD you will connect TDD with functional requirements Given, When, Then DDD the architecture of the application will fully reflect the business reality, so implementation of next requirements will be natural; no workarounds, no secret paths – only pure, unaffected and isolated model Building Blocks, Unambiguous Language, Strategic Design
It promises you that
you will eliminate bugs visible in production almost completely and generate no more code than required
Red-Green-Refactor, Unit testing
you will connect TDD with functional requirements
Given, When, Then
the architecture of the application will fully reflect the business reality, so implementation of next requirements will be natural; no workarounds, no secret paths – only pure, unaffected and isolated model
Building Blocks, Unambiguous Language, Strategic Design
Who creates it?
In the moment of creating their breakthrough concepts Ken Beck, Dan North, Eric Evans, Steve Freeman, Ivar Jacobson and other authors of mental frameworks already had many years of experience with different projects, people, fields and technologies. And what experience do those who mechanically apply the frameworks (thinking that it will change their lives) have? Later in this article I will come back to this question.
Paraphrasing a popular saying about building a house I may say that we build the first system for an enemy, the second one for a friend, and only the third one for ourselves. Think of your professional career and your first code that left much to be desired. Your then next code was so buttered with engineering stuff that you could hardly understand them. After some time, however, you finally reached the golden mean (it is interesting that you cannot halve the golden ratio) and the desired pragmatism.
According to The Transfer of Cognitive Skills by Singley, Anderson and Negative transfer errors in sequential cognitive skills by Woltz, Gardner, Bell, when you work on one project and start working on another, you usually experience transfer of learning, which means that skills you have mastered so far can be used in another project and they enable you to develop new useful skills more effectively. This transfer is the most desired option and it is called positive transfer. Yet, negative transfer is also possible, as mastering one skill may make it more difficult to master another.
All the authors of mental frameworks have migrated between various projects and various experiences. As a result, they have experienced the positive transfer of learning and gained particular knowledge, skills and expertise. In one moment they experienced what I call “enlightenment” (or more often, the eureka effect, the aha! effect or a paradigm shift). This is something that often happens by chance – sometimes in a dream – that triggers the entire chain of thought processes and our author experiences the enlightenment. Based on experiences, considerations and conclusions he arrives at one thought, for example Red-Green-Refactor. This thought brings order to the existing knowledge and imbues it with a deeper and more universal meaning. Technically speaking, this is inductive generalization (as illustrated in figure 1). Based on his expertise and by means of inductive reasoning the author builds a theory that generalizes individual experiences. Then he is ready to introduce a new mental framework to the world. She or he starts writing a book.
Figure 1 How a mental framework is created?
The consequences of induction
There is one problem with induction: it does not guarantee that the conclusions will be correct. That’s not a big deal, because a framework is not meant to be logically “correct”; it is meant to be “useful in a given context”. Now we have reached the point – the context is the key and it equals to experiences that served as the basis for new idea generalization. When the author wants to share his exploration with others and starts writing a book, new unexpected questions arise. How to use the framework in different contexts, which he is not familiar with? How to apply it to projects, architectures and technologies that he does not know? What does he do then? He uses deduction (figure 2).
Figure 2 How to use a framework in other contexts?
Based on his experiences and the framework that he came up with, he performs deductive reasoning and arrives at a conclusion that in this new context the framework should be used in one particular way. At this stage a different type of reasoning is essential. Inductive reasoning leads to a framework based on real experiences. Deductive reasoning specifies the framework through logical thinking, without confrontation with reality. Of course, this confrontation is possible, but the thing is that it may take years and the competition never sleeps.
In some cases deductive reasoning leads to correct conclusions; it other cases, it does not. Sometimes the conclusions only need to be brushed up. It seems that this is the case with Eric Evans’ book Domain-Driven Design and, in particular, the part about Strategic Design. When we take a look at Mr Evans’ activity, we will see that when he used DDD, he came to a conclusion that some ideas in the book should be made more precise or highlighted in a different way. Among some updates we will find: the introduction of the Domain Language Model Exploration Whirlpool process, particular stress put on the role of Bounded Context, a more precise meaning of a Domain Event, and a more precise relation between Bounded Contexts.
Who can use it?
What does the author do when he comes across a new and unknown context, in which his framework does not work? He resorts to his experiences, performs a small experiment, keeps track of the effects and then slightly changes the way his ideas can be applied. In most of the cases the author of a mental framework has so many experiences and so much expertise that he will know how to act in a new situation. In other words, this person is perfectly aware of the fact that the more dogmatically the framework will be used, the less profits he will have. Due to the fact that he uses his experience (which he always has at hand), he may almost instantly stop using his framework if he assumes that it makes no sense. Using his experience he chooses another solution and, possibly, modifies the framework.
Now imagine the following situation. A relatively inexperienced developer starts using *-Driven*. What are the effects to come? As long as he does the programming part according to the book and tutorials available in the Internet, everything runs smoothly. Due to the fact that we rarely create tutorial systems, problems will appear quite soon. The developer will face the following problem: how to correctly apply *-Driven* in a new and unknown context. Our developer does not have years of experience, making mistakes and achieving successes. He does not have the expertise, which he could use. He is left alone. Because he is pressed for time, he will do one or a few of the following things:
- asks someone more experienced (this is the best option);
- starts experimenting and will finally work out the correct approach (quite expensive option, but still reasonable);
- finds a solution (not necessarily correct) on the Internet;
- starts to apply the mental framework in a dogmatic way and will do it against all odds, even if it makes no sense;
- stops using the framework and reverts back to his old habits.
In three out of five cases the possible effect is discouragement – and no framework-related promise is fulfilled. You probably guess that it does not mean that *-Driven* does not work. The developer just does not have the necessary experience to correctly apply the idea. The typical feature of mental frameworks is that they are quite general. It would be difficult to write a detailed procedure to employ in any situation. I don’t even know if it is possible. Mental frameworks require interpretation. When you start using them and expect advantages only, you always have to interpret them in a given context. The basis of this interpretation is your own project experience.
Of course, there are also developers who have the experience required by a given mental framework. What is their approach? They usually have their own reflections and their own frameworks. Some of them, when they have a look at a new “revolutionary” *-Driven*, will find it interesting and realize that they already use elements of it. They will take what they want from the new framework, or they will even start using it as a whole, because it is always nice to use something that has neat and catchy names. The framework will enrich their resources and make their work more orderly.
Based on some loose observations I have arrived at some features that are characteristic of the developers that I have just described:
- they have more than ten years of experience;
- their experience is varied: different companies, different projects, different technologies, different people;
- they are able to create generalized concepts through generalization of some patterns that appear in individual and seemingly unrelated experiences;
- at the level of their methods of work they are able to precisely distinguish between interface and implementation, so they are more focused on the goal to achieve and not the means to employ to achieve it;
- they retrospectively evaluate their resources;
- they take time to expand their knowledge and develop their skills;
- in many cases they look for the causes of their failures in their conduct, not in the external environment;
- they are able not to grow attached to their tasks.
The last two features are usually the biggest challenge. To master them you usually need a very unusual coincidence or years of intensive and informed training. In our opinion, these features are the key element for every developer to enrich his mental framework, and even (with some effort included) introduce his own *-Driven* to the world. Without these two features a person gathers a critical mass of knowledge and skills that are sufficient to work comfortably. Such a person rarely pays attention to mental frameworks, because she usually knows better.
So, now do you know why *-Driven* will not change anything? Because when a developer does not have the necessary experience, the framework will not help – in fact, it will disturb. Such a person thinks that he develops professionally, but he actually gets more knowledge and less skills. He uses frameworks in a different way that he should, because he is not able to creatively interpret them in a given context. If a developer has the necessary experience, he already has his own frameworks designed or is not interested in them at all, because he thinks that in his case “it won’t work”.
What are the alternatives?
Definitely, *-Driven* certification of developers seems an attractive strategy of team development. I do not judge it. This is more a question of the strategy chosen and if it brings the expected results, there is nothing to talk about. In the folowing paragraphs I present some alternatives that you may find valuable.
Working on basic skills
If I was to summarize the entirety of software engineering in three words, these would be: responsibility, encapsulation, composition. Each of these three words can be the topic of a separate article or even a book. It’s a pity that in our struggle for *-Driven* we forget that these are probably the most important concepts that we should focus on. We often ask ourselves: what pattern to use? which mental framework to select? Instead, we should ask: what is the responsibility of the method, class, module? Responsibility, encapsulation, composition are the foundations of each of the software engineering techniques.
If you become skillful in using these concepts, the patterns will appear on their own. If you ignore them, you will soon end up with a parody of software engineering. At the same time these concepts are so general that irrespective of your experience you may always understand them on a deeper level.
Therefore, your first stage should be to concentrate on basic skills. When you master them, you can move on to the more complex ones. These are:
- using responsibility, encapsulation, composition;
- thinking from the perspective of the interface, i.e. “how will people use my component?”;
- working with code using techniques such as legibility, informativeness, simplicity, self-description and avoiding explicit use of patterns whenever possible;
- ability to ask questions that pinpoint “the essence” of a particular business; this “essence” is a model, but it does not mean classes and methods – it means a response to the question “how does this business really work?”
It is interesting that maybe the code created according to these rules is not “the best”, but it definitely is “easy to refactor”. In a relatively easy way we can modify its structure. This is the opposite of code that is difficult to refactor and which in some cases has to be written all over again when change is needed.
Another strategy that is more effective than the obsession with *-Driven* in the development of developers, is an informed management of skills in a team. Ages ago to get a job you had to enter into a Master-Journeyman relationship. The modern names for it are Mentor(Coach)-Apprentice. Currently, at the beginning each developer gets a pile of documents and simple, repeatable tasks to do. Sometimes he gets stuck in these tasks for months or years. In some cases it decreases his chances for success on the job market. I will venture a statement that this new job mutes his potential and does not help him develop. Looking at it from the point of view of skills development management, saying that each professional can be replaced by a finite number of students equates to saying that each CEO can be replaced by a finite number of advisors.
An organization that manages the skills of its developers, takes the following actions:
- it teaches the experienced engineers how to become mentors and coaches;
- it supports the new employees with a person who takes care of the development of their skills;
- it provides the developers with the possibility of working with different projects, technologies, clients and co-workers (of course, not simultaneously);
- it has a well-defined extra-technical profile of an employee and does not hire people who do not match it;
- when it employs a person, it treats him/her as a potential for development;
- it is oriented towards building mutual trust with the developers.
Replaceable human resources
There is at least one more approach based directly opposite to skills development management. Perhaps this the least welcome approach for the developers, but it is used even if the company does not declare it. This is why it is important to present it here. In this case the main goal is to build structures that enable the company to replace most of its developers in a fast and cheap way. Organizations that act this way do the following:
- They employ a specific number of highly experienced engineers, while the rest of the developers have average expertise and equally average financial expectations;
- A small group of people constructs system architecture in such a way that it can be further developed through repeatable patterns and the implementation cost is reduced to minimum;
- System architecture is adjusted to the expertise of the developers and the key changes and innovations are always introduced by the specialized group of the highly experienced and well-paid engineers;
As was mentioned this article is not intended to criticize anybody or anything. It is about priorities and searching for a balance. Being focused on our basic skills first, helps to use a *Driven* approach properly especially in an unknown context. I do really appreciate all those mental frameworks because they do great job. But I think if software development is driven by something, these are stakeholder needs and our common sense. Being driven by anything else might be harmful simplification.
About the Author
Michał Bartyzel is a BNS IT coach and consultant. He analyzes and develops psychology of programming models which help software developers work better. Currently he is working on conversation-patterns.com for software professionals. Contact: email@example.com, mbartyzel.blogspot.com, @MichalBartyzel
"Working on basic skills" section sound like (properly understood) DDD :P
happy to read long well thought through article
more of this kind of thinking please
Most of the *Driven* approaches are actually a bundling of good practices. Because a catchy name like TDD travels a lot faster and easier dev's can miss out on the individual good practices and concentrate on the bundle. Your basic skill of 'thinking from the perspective of the interface, i.e. “how will people use my component?"' is one of the practices promoted by TDD, but dev's sometimes fail to see this outside of TDD.
I also see a similar theme in practices like OO and FP. Both lack a definition and are actually a grouping of good practices and techniques. It's funny to see it happening again after so many times, but the current backlash against OO in favour of FP shows that we aren't really learning much.
Re: more of this kind of thinking please
While I totally appreciate where the author was going, it took a long time to get there; the definitions chosen seem too narrowly focussed; and the argument seems to suffer from some cognitive biases of its own.
Some specific comments:
"It would be difficult to write a detailed procedure to employ in any situation. I don’t even know if it is possible."
How about the scientific method, or perhaps entropic intelligence (www.ted.com/talks/alex_wissner_gross_a_new_equa...) ?
"If I was to summarize the entirety of software engineering in three words, these would be: responsibility, encapsulation, composition. Each of these three words can be the topic of a separate article or even a book. It’s a pity that in our struggle for *-Driven* we forget that these are probably the most important concepts that we should focus on."
I think this statement is a good example of taking a narrower, academic (or pure science) view / definition. Contemporary software engineering is far more than those concepts - most notably, the practical implications of engineering the software together with the delivery of the value it affords to the user / consumer (e.g., Continuous Delivery). Under this definition, it is far easier to see why the "*Driven" patterns (or any patterns for that matter, like Design Patterns) have emerged from the work in the field.
I agree with the author about understanding the basic concept and theory. As with the application of any technology (physical, patterns, etc.), it is the responsibility of the applier to understand the underlying laws of the problem domain, and the intentions of the technology, and then apply appropriately. Otherwise, cargo-cult behaviour abounds.
Some pragmatic quotes just for the heck of it:
"Experience without theory is blind, but theory without experience is mere intellectual play." -- Immanuel Kant
"In theory, theory and practice are the same; in practice they are not." -- Einstein
"No plan survives first contact with the enemy." -- Moltke the Elder
Refreshing and reassuring
At some point I started wondering if I was becoming outdated because of how many times it all ended up with "Maximize cohesion" and "Minimize coupling", along with some common sense and, of course, some new bits every now and then.
Thanks for that great insight!