Bedtime User Stories: Cowboys and Fairytales
David Longstreet , who identifies himself as "a software economist and international consultant," authored a paper last year claiming that Agile Software Development is a Fairy Tale and that it just tries to legitimise "cowboy" development. Geoff Slinker, author of the maverick software development model, used logical arguments to demonstrate the fallacies in David's thinking, and invited him to try again, taking a more rigorous approach.
From David Longstreet's 2007 article [pdf]:
Up to this point in time software development has been a Wild West endeavor.
... IT has been sloppy. There is nothing new with Agile, because it only tries to formalize sloppiness.
Geoff Slinker used logical arguments to demonstrate the fallacies in David's thinking. Like David, Geoff himself was initially skeptical of XP and even started writing a paper exposing XP flaws. However as he studied XP he began to see the value and started to try the practices on his own projects. Of David's claim that Agile legitimises sloppiness, Geoff countered that use of such a flawed argument is nothing more than an appeal to ridicule or spite.
David claimed that Agile doesn't value requirements and specifications; Geoff countered by looking at what, exactly, David wrote about User Stories in Agile:
Discussing pair programming Mr. [David] Longstreet states, "The idea is that one programmer writes code and the other programmer stands over his shoulder and watches for mistakes."
This is a complete falsehood. He goes on to say, "I am not sure what problem pair programming is trying to solve. Most of the issues with software development are related to incomplete requirements, not coding."
The first part of his statements on pair programming is a Straw Man. Also his statement that most issues are related to incomplete requirements is confusing cause and effect, and is an appeal to consequences of a belief.
David went on to say that:
Incomplete requirements are the biggest issue facing software development. I guess it is clear to the Agile folks that it is only logical to spend more time coding instead of cleaning up requirements or writing concise requirements in the first place.Geoff pointed out that this is the fallacy of confusing cause and effect, and concluded by inviting David to write a paper challenging Agile methods based on logic and citing sources.
Incomplete requirements...thats funny
Mapping from what I think the market needs to what the team can build consistent with my company's "big idea" takes a hell of a lot of dancing. The entire process in fraught with assumptions, guesses, judgment calls and compromises.
To translate from that world to one where "requirements are complete" is completely naive, in my opinion. For me the strength of agile is to get more people engaged on solving what the ambiguous requirements should be given market, sales, tech forces all acting together. The alternative is to attempt to encapsulate all that ambiguity in a PRD or a business case then use that as a foundation for building your firm and complete requirements.
Let me tell you how false a foundation that really is. Its so much more powerful to be able to zig and zag, try things and measure the result more often.
My take is that he views Agile Software Development as a threat to his business or his values. Ironically, his views on having meaningful measurement could likely work quite well within an agile process, and probably increase his overall business, IMHO.
Some merit, poorly put
I'm all for exposing some of the unhelpful practices, consultantspeak and myths that have sprung up around the Agile bandwagon, but the fact is the core idea works. I've never met a sane business sponsor who didn't accept that their requirements might be proven inaccurate, that technical issues come up from nowhere, that costs can be poorly estimated - all they want is simply to have no surprises when they do, and sometimes (although not all the time) agile promotes this better than a heavily change-controlled (change prevention) process.
I think Longstreet mixes practices designed to improve software quality and approaches designed to improve software delivery (a kind competence vs. attitude argument) and bends both to fit his slightly prejudiced point of view.
There's no doubt that misuse of the term agile is a problem. I'm getting a bit fed up of projects doing "agile" only to fail and then pretend they weren't doing "real agile" and that's why they failed.
It's the Bill Clinton defence. "I. did. not. have. agile. relations. with. that. project". Only at the post-implementation/failure review to admit, "While my answers were legally accurate, I did not volunteer information. Indeed, I did use a methodology that was not appropriate".
I provide two footnotes to support my position on pair programming. The first reference comes from Agile Software Development by Robert C. Martin. Martin writes while one person codes, “the other member of the pair watches the code being typed looking for errors and improvements.” Another reference is from Agile & Iterative Software Development A Managers Guide it reads, “The observer is doing a real time code review and perhaps thinking more broadly than the typist, considering test and so forth.” The Wikipedia definition of “Pair Programming reads, “One types in code while the other reviews each line of code as it’s typed in.”
I am not sure why Geoff is calling this a strawman argument. My argument does have some merit maybe it is not the argument Geoff wants me to make about pair programming. I think I need to add a sentence that reads, “Besides checking for errors the observer typically daydreams about the meaning of life and what they watched on TV last night.”
I love the idea where “one member controls the keyboard and the other controls the mouse.” (see page 13, Agile Software Development, Robert C. Martin). Perhaps this would work for one-armed programmers.
I am working on two new articles, “Agile and Jello: Two things you can’t get your hands around” and “Agile & Atkins Fads that leave you feeling empty.”
Re: Incomplete requirements...thats funny
Software developers know little about the core business they support. Throughout my career as a software professional, I have heard developers say, “Our users do not know what they want, and they are always changing their minds.” Another way to phrase this would be, “The core business does not know what they want, and they are always changing their mind.” Perhaps it is a mistake to assume the core business has the ability to explain what it wants a software application to do and not do.
This problem is not unique to software development and I would encourage all to read and study how industrial designers study customers. If customers are not capable of articulating what they want and what is missing, then how should the requirements process work?
A common Agile practice is sitting in a conference room with a customer and asking, “What do you want?” Sitting in a conference room asking questions may not be a useful way to gather requirements, and it certainly should not be the only way. This has nothing to do with how smart or how good you are at asking questions. Some developers will code what they thought the customer said and then show it to the customer only to find out thats not what was meant. This is commonly known as a code/test iteration but is really trial and error method to gather requirements.
Instead developers need to study the core business and learn the business. They need to stop sitting in conference rooms waiting to be told what to code. They need to go where their customers work.
The root cause of raining requirements is the software developers lack of knowledge related to the core business.
A member wrote, “Mr. Longstreet came here at our invitation.” Another member wrote, “I also deplore name-calling and threats. I also try to address/criticize ideas rather than the people who wrote them.”
If anyone would like to follow the mudslinging or name calling directed my way I would encourage them to visit the Extreme Programming Yahoo Group. You can read the posts and make your own determinations. Like my paper, you do not need to rely on what others say because you can draw your own conclusions.
I do not mind the name calling or the 100's of nasty emails I get from Agile folks. Often the one guy who objects to a popular or trendy idea turns out being right. No doubt Agile is the fad of the day.
These type of posts by Dave Rooney are boring, they are boring to respond to and boring to read. Dave does not know my motivation, my values and he does not know me either.
Re: Some merit, poorly put
Response to Geoff's Comments
1. Pair ProgrammingGeoff writes my opinions are "complete falsehoods" Kinda of harsh since in my paper I provide footnotes with page numbers, books etc. (see comments in previous post "Strawman?"
2. Code/Test = Trial and Error Code/Test sounds more dressed up and better than trial and error.
2. Customer Practices - Geoffs writes that what I write is hearsay I am quoting from the XP Extreme Programming Group (this is also footnoted in the paper). Geoff gets the quotes right, so if anyone wants to find out who these folks are making these statements they only need to search them out. Some are authors of books and papers.
3. Geoff uses appeal to authority a bunch of times. I had to look up the definition of appeal to authority. I would suggest he check out my client list at www.SoftwareMetrics.Com/client.htm. I also teach graduate level statistics and quantitative analysis to those getting MBA's and graduate psychology degrees. I speak at conferences around the world and I referee journals and conference proceedings for the Academy of Management.
Now I am no dairy farmer, but I think my background and experience makes me qualified to speak on the topic of software development (even Agile). If I started talking trash on the GB Packers, Wisconsin Cheese, or Dairy Cows, then I guess the Appeal of Authority would work.
I hate to burst Geoff's logical bubble, but even those that may disagree with him can still be an authority in the field. I am puzzled why he uses this argument on several occasions. It is like being in a debate and the person saying, well, you are just not qualified to talk on this subject...what? I guess only those people that agree with Agile (and Geoff) are qualified to speak on the topic of Agile.
4. Dude, Geoff, buddy... I worked hard on a lot of the paper and you and so many other Agile types don't even bother to comment on the best parts. I am hurt. I even went and read Ogunnaike ( to to Agilistas as Ogannaike) book on Process Dynamics Modeling & Control, I even explain what is meant by "empirical modeling v. theory & how Agile misuses the terms. I even use some big words like "stochastic." I know it is not logic, but I provide a mathematical proof from set theory.
I even write about Queuing Theory and quote from some books and papers. I think my paper has 20 footnotes and references.
I think people are capable of reading the paper and making determinations if the paper has merit or if I have enough background to comment on the topic of software development and specifically Agile.
I know if is not popular to speak against such a wonderful fad as Agile, but often the one opposing voice is the right voice. Clearly Agile is the fad of the day for software development. This too shall pass.
For those that may be wondering, a Straw Man argument is a distortion of an issue and then an attack on the distortion and not the original issue.
"The idea is that one programmer writes code and the other programmer stands over his shoulder and watches for mistakes."
The distortion is "stands over his shoulder" which invokes images which many would find ridiculous.
David says he should add:
"Besides checking for errors the observer typically daydreams about the meaning of life and what they watched on TV last night."
This is the very problem with the original paper. These are the very types of statements that make it difficult to take the arguments serious.
It seems that David truly has beliefs that Agile is some sort of scam because it seems evident in such mocking as "Perhaps this would work for one-armed programmers."
Since David would prefer to add his day dreaming comment to the his definition of pair programming let's look at the rest of the definition from Wikipedia:
Pair programming is a software development technique in which two programmers work together at one keyboard. One types in code while the other reviews each line of code as it's typed in. The person typing is called the driver. The person reviewing the code is called the observer or navigator. The two programmers switch roles frequently.
While reviewing, the observer also considers the strategic direction of the work, coming up with ideas for improvements and likely future problems to address. This frees the driver to focus all of his or her attention on the "tactical" aspects of completing the current task, using the observer as a safety net and guide.
I am not the author of XP nor Agile. I am not going to defend Agile, however I am going to defend my critique of David's paper.
I stand by my evaluation that the paper needs improvement by removing logical fallacies and polishing the paper with concise and accurate details, facts, and statements.
Re: Response to Geoff's Comments
I think the two programmers are supposed to touch
hands like professional wrestles do in a tag team-wrestling match.
David, why do you make such a statement? If it is supposed to be humorous then I missed it. I read the paper as if it is to be a professional critique of XP and Agile.
David, please prove the following:
Code/Test = Trial and Error
And after that please show how your proposed method of development does not experience any trial and error aspects.
Geoff uses appeal to authority a bunch of times. I had to look up the definition of appeal to authority. I would suggest he check out my client list at www.SoftwareMetrics.Com/client.htm. I also teach graduate level statistics and quantitative analysis to those getting MBA's and graduate psychology degrees. I speak at conferences around the world and I referee journals and conference proceedings for the Academy of Management.
David, there you go again appealing to authority. It's not that your findings are right or wrong, it's that the argument doesn't hold solely because you say so.
I hate to burst Geoff's logical bubble, but even those that may disagree with him can still be an authority in the field. I am puzzled why he uses this argument on several occasions.
I don't have a bubble. ;-)
We both have declared ourselves authorities in software development. But that declaration alone is not sufficient.
David, I am sure you worked very hard on the paper. I feel most know I do not intend to cause you any hurt. Maybe I should take a new approach. David, I suggest you try this, take your paper and submit it to the IEEE or some refereed publication in the Software Industry. I wonder how well your style and references would hold up.
David says, "often the one opposing voice is the right voice".
Sometimes it is true that the voice opposing the common view is correct. This could be taken as an argument in favor of Agile since Agile Software development is in the minority the last time I checked.
Re: Response to Geoff's Comments
Code/test is a method to gather requirements. It is really a <em>trial and error method</em>. You build something, show it someone and ask is this what you want. The jest of the argument is to sit and to be told what to code, then show it someone to see if it was correct. Sounds like, smells like trial and error. This creates another job in IT called <em>"software waiter</em>" they just take orders.
I point out in the article, in all my writings and talks the lack of business knowledge in IT is the real source of poor requirements. Not only do <em>IT folks lack business knowledge they lack the skills to study the core business using ethnological techniques</em>.
Once upon a time, the folks who drilled oil wells were the folks that were "engineers." As the petroleum industry matured the engineers used science to improve the techniques for drilling and especially finding oil. Now the people who do the drill'n and the engineering work are not the same folks; however the people who write the code and "software engineers" are often the same.
Check out any other discipline like Mechanical Engineering, Electrical Engineering, Law, Medicine, they all specialize along some domain. Software developers are generalists right now and that is starting to change. As the industry matures we will see more and more specialization along an industry. We also see a division of craftsmen and engineer. We have seen this over and over again. Agile wants to hold onto the idea of craftsmen.This is called the division of labor (<em>see Adam Smith</em>) and every industry has experienced this same thing.
Agilistas want to hold onto the craftsmen idea and focus on the code. In your own blog the article, What is Necessary to Develop Software? look at the number of times you use the word code, program and how many time you use the word, "customer" or "core business." What is necessary to develop software is a clear understanding the business and business problem -FIRST-- where is the customer? In another article you write, "Customer Input, Who needs it" you write about the customer from a coding/technical perspective not from a business perspective. Again you focus on technology and code not on the business problem.
I have clients that see giant leaps in productivity, quality and profitability when they learn & study the core business and do not wait to be told what to do. The stop focusing on the code and focus on the business problem. They adopt techniques that industrial designers use. As I point out, often customers lack the vocabulary to tell you what is wrong and what is missing. You can't rely on customers (the core business) to tell you what they want. You have to go the " customer jungle" and study them. My clients have seen leaps in productivity from 40 hours/function point to less than 5 hours per function point.
The burden of proof is upon Agile to show it is a good idea (<em>see hypothesis testing</em>). It is not up to me to prove Agile wrong. It is up to Agilistas to prove it right. Martin Folwer writes on his own blog it is basically impossible to measure software development. How cool is this. If you can't measure software development you can't prove Agile right or wrong.
Here is my battle cry for IT, <strong><em>LEARN THE BUSINESS! </em></strong>If you don't learn the business, you cannot participate in strategic planning and especially release management. You will have no idea what should be in a release or not in a release of software. If you sit in a conference room and wait to be told what to code, then you are nothing more than a <em>software waiter (a.k.a. Agile).</em>
PS. Let me put on my academic hat. I referee papers for the Academy of Management. I even have a book published by IEEE Computer Society. I know the required writing style for academic publications. The paper was not written as an academic paper.
Using the academic standard, I reject Agile because there are not a series of academic papers with quantitative data proving this is a good technique. There are no p values and there is no evidence. I fail to accept your hypothesis because you have not proved it using the scientific method and peer review process. I would recomend you not go this route with your argument.
Agile should be the ones writing journal articles proving that Agile is a good idea or not a good idea.It would take a couple of years to get something published and then software development will be into some other fad.
Future article ideas,
<em>Agile and Jello: Things you Can't Get Your Hands Around</em>
<em>Agile and Atkins Fads that leave you feeling empty
<em>Learn Agile and become a software waiter.</em>
Geoff I really think you are misusing the appeal to authority idea and many of your other logical arguments are misapplied. Those that want to learn more please see Mortimer Adler in regards to "Appeal to Authority" -- I would recommend his books, "Ten Philosophical Mistakes", "How to Speak and How to Listen","How to Read a Book." and "Aristole for Everybody." <em>Let's do try to stay away from being to intellectual because frankly I find it extremely boring.</em> I am not certain you are an authority on logical argument. People can make up their own minds whether you are right or wrong on the logical arguments.
Agile is not the fad de jour? OK if you think so.
While this is entertaining and fun, I do need to get back to work.
Pair Programming & Professional Wrestling
Oh, I forgot to say something about Pair Programming.
I am not really sure how to think about Pair Programming because many people write me that nobody really does pair programming. I have to think pair programming was a management idea because they did not have two computers for two programmers. The manager thought, "Hey pair program."
Yes, I do write that pair programming is similar to professional tag team wrestling and programmers have to touch hands as they switch roles. I think Agile programmers need to start adopting names like professional wrestlers too.
Here are some potential names (I use some spanish names because I know Agilstas have an appreciation for Spanish/Latin). Excuse my bad Spanish, but here goes.
El Can de Coder (I think this translates as the dog coder)
La Bruja Coder (the Witch Coder)
El Programmer Volador (The flying programmer)
El Coder del Estilo Diferente (The coder with a different style)
El Duende que Codes (The ghost that codes)
Buffy the Bug Detector
The Prodigy Programmer
Geoff try not to take this argument or yourself too serious.
Re: Response to Geoff's Comments
Code/test is a method to gather requirements. I am not the author of XP, nor Test Driven Development, nor Crystal Clear, or any other Agile method that you seem to be referring to. Is this "Iteration Code/Test" you speak of supposed to be reference to Test First Programming or Test Driven Development? If so then your statement would be accurate if it said "Code/test is a method to design code at the implementation level."
Sounds like, smells like... still does not make something true. Oh well, am I Maverick and live on the frontier of Agile and every other methodology. I do not make any money from their books, I am not an Agile consultant, and so my interest in Agile is purely from an "application" point of view. I still have the challenge for the authors of the Agile methods you question to precisely define what it is their method does, how it does it, and what is to be expected from doing it. I think they would argue that the current body of knowledge has defined their processes and the answers can be found in their books, papers, and statements. For instance, who owns XP and thus IMO is the only one that can say what XP really is? In my opinion it is Kent Beck and Kent Beck only. Who owns Crystal Clear? Alistair Cockburn. Who owns Maverick? I do! Any one else's statements are mere opinion in my opinion.
I am glad you have read some of my posts. Thank you.
As for what is needed for code, what I am describing is the simplest set of "things" needed to write code. What you are concerned with is developing software that meets the needs of a specific customer. That is something different all together. What do you need to meet your concerns? I will not try to address that here but you need a lot more than a computer and something or someone to generate instructions for the computer to execute!
I have clients that see giant leaps in productivity, quality and profitability when they learn & study the core business and do not wait to be told what to do.
First, you have declared that Agile is Code and Test which is Trial and Error wich is sit and wait to be told what to do next. I still say that this argument is invalid.
Second, I have never worked anywhere where the programmers sit and wait to be told what to do next. Never in all my years. You paint a picture that is quite distorted to me.
Here is a straight up question.
Have you ever been at an XP development shop where the developers sat and waited as you describe and then you introduced your method and experienced the leap in productivity that you describe?
Have you ever participated in gathering requirements as defined and described by Alistair Cockburn? How does Alistair's method of Use Cases ignore the customer and create a "software waiter" situation?
As for my posts on Digerati Illuminatus (tech.groups.yahoo.com/group/digerati-illuminatus/) you criticize my postings because I do not take the business perspective. Well sorry but I have to say, DUH! That wasn't the point. I will do this, propose some topics and I will work with you and write a blog posting including the business perspective and post it to my site and I will be your co-author on the paper.
It is not up to me to prove Agile wrong. It is up to Agilistas to prove it right.
Well, you started this with the fervor as if to say you are protecting all businesses that depend on software from the latest scam called Agile. If your critique is to be taken serious then it should be of the highest professional quality. You state the paper was not written as an academic paper. So what is it written as? A statement of facts? An attack? An appeal? What?
Here is my battle cry for IT, LEARN THE BUSINESS! If you don't learn the business, you cannot participate in strategic planning and especially release management.
Is this scaleable? I imagine you are describing a situation where a consultant that comes into busineses with very focused and specific needs which results in a small team of 1 - 4 developers that are hired to "attack" the problems. The companies I have worked at do not want ALL of the developers participating in strategic planning and release management. There may be hundreds of programmers at the company. There are near one hundred where I work now!. So are your recommendations wholesale usable by me? No. So what types of business, what size of business can use and sustain having all of development doing strategic planning?
David, you admit you didn't even know what many of the logical fallacies are and now you have a recommended reading list! WOW! I am not certain you are an authority on logical argument. I am certain you are not an authority on Agile software development, but you are learning and reading, and thinking about it. Try a little application and experimentation, yes I know this is emperical. You do have first hand experience with the Agile practices, right?
David, I appreciate your comments and your concerns.
For the record, I do not make any money from Agile Software Development. I have no books on the topic. I do not consultant on the topic. I do not travel and sell it. You really need to get a discussion with the authors and I am not one of them. I have critiqued your arguments and I find them unsound. That's all. If you present accurate representations of Agile practices I feel your arguments would be "better".
David, I am sincere in that I have nothing against you. If there are any personal attacks in my post I appologize. Even my arguments fall victim to common fallacies.
You sure like P.S.'s alot!
Geoff try not to take this argument or yourself too serious.
Okay. I am not a serious person on a personal level. If the paper wasn't meant to be taken seriously then first, sorry for the time I have taken from you, the community, and even myself.
If the paper isn't serious then I feel it was just a scheme to attract attention to your website and business and thus I have been victim of an adverstising ploy. A little controversy goes a long way.
Consider the topic closed and on the "not to be taken seriously" list.
Re: Response to Geoff's Comments
For the record I guess I am an agilista, even though I think the term misrepresents the spirit of Agile. I think of myself as more of a pragmatist and not particularly dogmatic (but I was raised as a Catholic, so I might be wrong).
I've been using agile for several years, both successfully and unsuccessfully. On the unsuccessful projects ,always the most interesting, the problems have been with a flawed implementation of the concepts, not the core concepts. On the plus side, Agile's insistence on constant reflection and improvement, IF PRACTICED, can provide a mechanism to correct problems.
After reading David Longstreet's article several times, and both supporting and critical reviews of it, I'm left to conclude that Mr. Longstreet just isn't interested in learning what Agile really is. I'm not sure why, but anyone that spends some time to learn what agile is all about will recognize the fallacy of his arguments. Sure agile isn't a "silver bullet" ,to quote Fred Brooks, but it can be very helpful in many circumstances. Unfortunately, many will take the easy way out and accept his criticisms as fact. Me, I like options, or to put it another way, more tools. Agile techniques provide more tools for me to put in my bag, more options to have available when tackling those "opportunities disguised as unsolvable problems".
This is all a long-winded, and admittedly not a particularly agile, introduction to another discussion that came up on the google search I mentioned at the beginning of my post - www.targetprocess.com/blog/2007/06/erroneous-an.... I found it interesting, maybe others will too (but then again I'm a commie pinko agilista :>).
Finally, I think Mr. Longstreet does endorse the concept of "decide for yourself". By all means do read his article. He does a good job of raising arguments that come up again and again against agile techniques. But before you decide, do some homework on what leading advocates of agile and lean techniques say too (Mary & Tom Poppendick, Jean Tabaka, Mike Cohen, Martin Fowler, Kent Beck, Alistair Cockburn, Hubert Smits). There are lots of good ideas out there. Pick the ones that make sense to you, try them, but by all means continually evaluate whether they're working for you.