Bedtime User Stories: Cowboys and Fairytales

As a product manager and scrum product owner I find it funny that folks blame incomplete requirements as "the biggest issue facing software development". In some ways I agree but as somebody who's had responsibility for defining a product backlog based on several different inputs: user feedback, internal innovation, technical/market dynamics etc etc I realize that its a real art.

Mapping from what I think the market needs to what the team can build consistent with my company's "big idea" takes a hell of a lot of dancing. The entire process in fraught with assumptions, guesses, judgment calls and compromises.

To translate from that world to one where "requirements are complete" is completely naive, in my opinion. For me the strength of agile is to get more people engaged on solving what the ambiguous requirements should be given market, sales, tech forces all acting together. The alternative is to attempt to encapsulate all that ambiguity in a PRD or a business case then use that as a foundation for building your firm and complete requirements.

Let me tell you how false a foundation that really is. Its so much more powerful to be able to zig and zag, try things and measure the result more often.

Jim
www.wattf.com/

David has dropped by twice now on the XP Yahoo group. Initially, he was somewhat engaging, but he degraded quickly into a classic troll. People tried to have a reasonable discussion with him (and I do honestly mean reasonable), but in the end he wasn't interested.

My take is that he views Agile Software Development as a threat to his business or his values. Ironically, his views on having meaningful measurement could likely work quite well within an agile process, and probably increase his overall business, IMHO.

Dave Rooney
Mayford Technologies

Longstreet's paper makes some good, and valid, points, notably that researching the business you're in will help enormously in eliciting better requirements. It's just a shame they are wrapped up in 20 pages of tedious self-indulgent blather.

I'm all for exposing some of the unhelpful practices, consultantspeak and myths that have sprung up around the Agile bandwagon, but the fact is the core idea works. I've never met a sane business sponsor who didn't accept that their requirements might be proven inaccurate, that technical issues come up from nowhere, that costs can be poorly estimated - all they want is simply to have no surprises when they do, and sometimes (although not all the time) agile promotes this better than a heavily change-controlled (change prevention) process.

I think Longstreet mixes practices designed to improve software quality and approaches designed to improve software delivery (a kind competence vs. attitude argument) and bends both to fit his slightly prejudiced point of view.

There's no doubt that misuse of the term agile is a problem. I'm getting a bit fed up of projects doing "agile" only to fail and then pretend they weren't doing "real agile" and that's why they failed.

It's the Bill Clinton defence. "I. did. not. have. agile. relations. with. that. project". Only at the post-implementation/failure review to admit, "While my answers were legally accurate, I did not volunteer information. Indeed, I did use a methodology that was not appropriate".

I would encourage you to read the article for yourself and make your own determination regarding the merits of the article and not listen to what others say or write about the article. www.SoftwareMetrics.Com/Agile

I provide two footnotes to support my position on pair programming. The first reference comes from Agile Software Development by Robert C. Martin. Martin writes while one person codes, “the other member of the pair watches the code being typed looking for errors and improvements.” Another reference is from Agile & Iterative Software Development A Managers Guide it reads, “The observer is doing a real time code review and perhaps thinking more broadly than the typist, considering test and so forth.” The Wikipedia definition of “Pair Programming reads, “One types in code while the other reviews each line of code as it’s typed in.”

I am not sure why Geoff is calling this a strawman argument. My argument does have some merit maybe it is not the argument Geoff wants me to make about pair programming. I think I need to add a sentence that reads, “Besides checking for errors the observer typically daydreams about the meaning of life and what they watched on TV last night.”

I love the idea where “one member controls the keyboard and the other controls the mouse.” (see page 13, Agile Software Development, Robert C. Martin). Perhaps this would work for one-armed programmers.

I am working on two new articles, “Agile and Jello: Two things you can’t get your hands around” and “Agile & Atkins Fads that leave you feeling empty.”

David Longstreet
Software Economist
David@SoftwareMetrics.Com
www.SoftwareMetrics.Com/Agile

Customers often lack the vocabulary to explain what’s wrong and especially what is missing. Software developers need to start learning ethnological techniques and study their customers instead of sitting in a conference room waiting to be told what to do.

Software developers know little about the core business they support. Throughout my career as a software professional, I have heard developers say, “Our users do not know what they want, and they are always changing their minds.” Another way to phrase this would be, “The core business does not know what they want, and they are always changing their mind.” Perhaps it is a mistake to assume the core business has the ability to explain what it wants a software application to do and not do.

This problem is not unique to software development and I would encourage all to read and study how industrial designers study customers. If customers are not capable of articulating what they want and what is missing, then how should the requirements process work?

A common Agile practice is sitting in a conference room with a customer and asking, “What do you want?” Sitting in a conference room asking questions may not be a useful way to gather requirements, and it certainly should not be the only way. This has nothing to do with how smart or how good you are at asking questions. Some developers will code what they thought the customer said and then show it to the customer only to find out thats not what was meant. This is commonly known as a code/test iteration but is really trial and error method to gather requirements.

Instead developers need to study the core business and learn the business. They need to stop sitting in conference rooms waiting to be told what to code. They need to go where their customers work.

The root cause of raining requirements is the software developers lack of knowledge related to the core business.

David Longstreet
Software Economist
David@SoftwareMetrics.Com
www.SoftwareMetrics.Com/Agile

Like so many things Agile, Dave Rooney is either confused or intentionally misleading. I was invited to the XP Extremme Programming group hosted on Yahoo. After numerous naming calling and mudsling was directed my way the moderator wrote, “I’m not sure what started this particular round of name calling, but lets stop it. NOW”

A member wrote, “Mr. Longstreet came here at our invitation.” Another member wrote, “I also deplore name-calling and threats. I also try to address/criticize ideas rather than the people who wrote them.”

If anyone would like to follow the mudslinging or name calling directed my way I would encourage them to visit the Extreme Programming Yahoo Group. You can read the posts and make your own determinations. Like my paper, you do not need to rely on what others say because you can draw your own conclusions.

I do not mind the name calling or the 100's of nasty emails I get from Agile folks. Often the one guy who objects to a popular or trendy idea turns out being right. No doubt Agile is the fad of the day.

These type of posts by Dave Rooney are boring, they are boring to respond to and boring to read. Dave does not know my motivation, my values and he does not know me either.

David Longstreet
Software Economist
www.SoftwareMetrics.Com/Agile
David@SoftwareMetrics.Com

Let's review,

"self-indulgent blather" -- nice.

Again, read the paper for yourself and make your own determination.
www.SoftwareMetrics.Com/Agile

David Longstreet
Software Economist
www.SoftwareMetrics.Com

I will tackle Geoff's comments one by one. Here is the paper that Geoff talks about but fails to include a hyperlink to www.SoftwareMetrics.Com/Agile.

1. Pair ProgrammingGeoff writes my opinions are "complete falsehoods" Kinda of harsh since in my paper I provide footnotes with page numbers, books etc. (see comments in previous post "Strawman?"

2. Code/Test = Trial and Error Code/Test sounds more dressed up and better than trial and error.

2. Customer Practices - Geoffs writes that what I write is hearsay I am quoting from the XP Extreme Programming Group (this is also footnoted in the paper). Geoff gets the quotes right, so if anyone wants to find out who these folks are making these statements they only need to search them out. Some are authors of books and papers.

3. Geoff uses appeal to authority a bunch of times. I had to look up the definition of appeal to authority. I would suggest he check out my client list at www.SoftwareMetrics.Com/client.htm. I also teach graduate level statistics and quantitative analysis to those getting MBA's and graduate psychology degrees. I speak at conferences around the world and I referee journals and conference proceedings for the Academy of Management.

Now I am no dairy farmer, but I think my background and experience makes me qualified to speak on the topic of software development (even Agile). If I started talking trash on the GB Packers, Wisconsin Cheese, or Dairy Cows, then I guess the Appeal of Authority would work.

I hate to burst Geoff's logical bubble, but even those that may disagree with him can still be an authority in the field. I am puzzled why he uses this argument on several occasions. It is like being in a debate and the person saying, well, you are just not qualified to talk on this subject...what? I guess only those people that agree with Agile (and Geoff) are qualified to speak on the topic of Agile.

4. Dude, Geoff, buddy... I worked hard on a lot of the paper and you and so many other Agile types don't even bother to comment on the best parts. I am hurt. I even went and read Ogunnaike ( to to Agilistas as Ogannaike) book on Process Dynamics Modeling & Control, I even explain what is meant by "empirical modeling v. theory & how Agile misuses the terms. I even use some big words like "stochastic." I know it is not logic, but I provide a mathematical proof from set theory.

I even write about Queuing Theory and quote from some books and papers. I think my paper has 20 footnotes and references.

I think people are capable of reading the paper and making determinations if the paper has merit or if I have enough background to comment on the topic of software development and specifically Agile.
www.SoftwareMetrics.Com/Agile

I know if is not popular to speak against such a wonderful fad as Agile, but often the one opposing voice is the right voice. Clearly Agile is the fad of the day for software development. This too shall pass.

David Longstreet
Software Economist
www.SoftwareMetrics.Com

digerati-illuminatus.blogspot.com/2008/02/agile...


For those that may be wondering, a Straw Man argument is a distortion of an issue and then an attack on the distortion and not the original issue.

"The idea is that one programmer writes code and the other programmer stands over his shoulder and watches for mistakes."

The distortion is "stands over his shoulder" which invokes images which many would find ridiculous.

David says he should add:
"Besides checking for errors the observer typically daydreams about the meaning of life and what they watched on TV last night."

This is the very problem with the original paper. These are the very types of statements that make it difficult to take the arguments serious.

It seems that David truly has beliefs that Agile is some sort of scam because it seems evident in such mocking as "Perhaps this would work for one-armed programmers."

Since David would prefer to add his day dreaming comment to the his definition of pair programming let's look at the rest of the definition from Wikipedia:

Pair programming is a software development technique in which two programmers work together at one keyboard. One types in code while the other reviews each line of code as it's typed in. The person typing is called the driver. The person reviewing the code is called the observer[1] or navigator. The two programmers switch roles frequently.

While reviewing, the observer also considers the strategic direction of the work, coming up with ideas for improvements and likely future problems to address. This frees the driver to focus all of his or her attention on the "tactical" aspects of completing the current task, using the observer as a safety net and guide.

I am not the author of XP nor Agile. I am not going to defend Agile, however I am going to defend my critique of David's paper.

I stand by my evaluation that the paper needs improvement by removing logical fallacies and polishing the paper with concise and accurate details, facts, and statements.

digerati-illuminatus.blogspot.com/2008/02/agile...



Pair Programming



I think the two programmers are supposed to touch
hands like professional wrestles do in a tag team-wrestling match.



David, why do you make such a statement? If it is supposed to be humorous then I missed it. I read the paper as if it is to be a professional critique of XP and Agile.



David, please prove the following:



Code/Test = Trial and Error



And after that please show how your proposed method of development does not experience any trial and error aspects.



Geoff uses appeal to authority a bunch of times. I had to look up the definition of appeal to authority. I would suggest he check out my client list at www.SoftwareMetrics.Com/client.htm. I also teach graduate level statistics and quantitative analysis to those getting MBA's and graduate psychology degrees. I speak at conferences around the world and I referee journals and conference proceedings for the Academy of Management.



David, there you go again appealing to authority. It's not that your findings are right or wrong, it's that the argument doesn't hold solely because you say so.



I hate to burst Geoff's logical bubble, but even those that may disagree with him can still be an authority in the field. I am puzzled why he uses this argument on several occasions.



I don't have a bubble. ;-)



We both have declared ourselves authorities in software development. But that declaration alone is not sufficient.



David, I am sure you worked very hard on the paper. I feel most know I do not intend to cause you any hurt. Maybe I should take a new approach. David, I suggest you try this, take your paper and submit it to the IEEE or some refereed publication in the Software Industry. I wonder how well your style and references would hold up.



David says, "often the one opposing voice is the right voice".



Sometimes it is true that the voice opposing the common view is correct. This could be taken as an argument in favor of Agile since Agile Software development is in the minority the last time I checked.

Geoff,

Code/test is a method to gather requirements. It is really a <em>trial and error method</em>. You build something, show it someone and ask is this what you want. The jest of the argument is to sit and to be told what to code, then show it someone to see if it was correct. Sounds like, smells like trial and error. This creates another job in IT called <em>"software waiter</em>" they just take orders.

I point out in the article, in all my writings and talks the lack of business knowledge in IT is the real source of poor requirements. Not only do <em>IT folks lack business knowledge they lack the skills to study the core business using ethnological techniques</em>.

Once upon a time, the folks who drilled oil wells were the folks that were "engineers." As the petroleum industry matured the engineers used science to improve the techniques for drilling and especially finding oil. Now the people who do the drill'n and the engineering work are not the same folks; however the people who write the code and "software engineers" are often the same.

Check out any other discipline like Mechanical Engineering, Electrical Engineering, Law, Medicine, they all specialize along some domain. Software developers are generalists right now and that is starting to change. As the industry matures we will see more and more specialization along an industry. We also see a division of craftsmen and engineer. We have seen this over and over again. Agile wants to hold onto the idea of craftsmen.This is called the division of labor (<em>see Adam Smith</em>) and every industry has experienced this same thing.

Agilistas want to hold onto the craftsmen idea and focus on the code. In your own blog the article, What is Necessary to Develop Software? look at the number of times you use the word code, program and how many time you use the word, "customer" or "core business." What is necessary to develop software is a clear understanding the business and business problem -FIRST-- where is the customer? In another article you write, "Customer Input, Who needs it" you write about the customer from a coding/technical perspective not from a business perspective. Again you focus on technology and code not on the business problem.

I have clients that see giant leaps in productivity, quality and profitability when they learn & study the core business and do not wait to be told what to do. The stop focusing on the code and focus on the business problem. They adopt techniques that industrial designers use. As I point out, often customers lack the vocabulary to tell you what is wrong and what is missing. You can't rely on customers (the core business) to tell you what they want. You have to go the " customer jungle" and study them. My clients have seen leaps in productivity from 40 hours/function point to less than 5 hours per function point.

The burden of proof is upon Agile to show it is a good idea (<em>see hypothesis testing</em>). It is not up to me to prove Agile wrong. It is up to Agilistas to prove it right. Martin Folwer writes on his own blog it is basically impossible to measure software development. How cool is this. If you can't measure software development you can't prove Agile right or wrong.

Here is my battle cry for IT, <strong><em>LEARN THE BUSINESS! </em></strong>If you don't learn the business, you cannot participate in strategic planning and especially release management. You will have no idea what should be in a release or not in a release of software. If you sit in a conference room and wait to be told what to code, then you are nothing more than a <em>software waiter (a.k.a. Agile).</em>

David Longstreet

Software Economist

www.SoftwareMetrics.Com

PS. Let me put on my academic hat. I referee papers for the Academy of Management. I even have a book published by IEEE Computer Society. I know the required writing style for academic publications. The paper was not written as an academic paper.

Using the academic standard, I reject Agile because there are not a series of academic papers with quantitative data proving this is a good technique. There are no p values and there is no evidence. I fail to accept your hypothesis because you have not proved it using the scientific method and peer review process. I would recomend you not go this route with your argument.

Agile should be the ones writing journal articles proving that Agile is a good idea or not a good idea.It would take a couple of years to get something published and then software development will be into some other fad.

P.S.S

Future article ideas,

<em>Agile and Jello: Things you Can't Get Your Hands Around</em>

<em>Agile and Atkins Fads that leave you feeling empty

</em>

<em>Learn Agile and become a software waiter.</em>

P.S.S.S

Geoff I really think you are misusing the appeal to authority idea and many of your other logical arguments are misapplied. Those that want to learn more please see Mortimer Adler in regards to "Appeal to Authority" -- I would recommend his books, "Ten Philosophical Mistakes", "How to Speak and How to Listen","How to Read a Book." and "Aristole for Everybody." <em>Let's do try to stay away from being to intellectual because frankly I find it extremely boring.</em> I am not certain you are an authority on logical argument. People can make up their own minds whether you are right or wrong on the logical arguments.

P.S.S.S.S

Agile is not the fad de jour? OK if you think so.

While this is entertaining and fun, I do need to get back to work.

Cheers...

Oh, I forgot to say something about Pair Programming.

I am not really sure how to think about Pair Programming because many people write me that nobody really does pair programming. I have to think pair programming was a management idea because they did not have two computers for two programmers. The manager thought, "Hey pair program."

Yes, I do write that pair programming is similar to professional tag team wrestling and programmers have to touch hands as they switch roles. I think Agile programmers need to start adopting names like professional wrestlers too.

Here are some potential names (I use some spanish names because I know Agilstas have an appreciation for Spanish/Latin). Excuse my bad Spanish, but here goes.

El Can de Coder (I think this translates as the dog coder)

La Bruja Coder (the Witch Coder)

El Programmer Volador (The flying programmer)

El Coder del Estilo Diferente (The coder with a different style)

El Duende que Codes (The ghost that codes)

Buffy the Bug Detector

The Prodigy Programmer

Geoff try not to take this argument or yourself too serious.

David Longstreet

Software Economist

www.SoftwareMetrics.Com

David,


Code/test is a method to gather requirements. I am not the author of XP, nor Test Driven Development, nor Crystal Clear, or any other Agile method that you seem to be referring to. Is this "Iteration Code/Test" you speak of supposed to be reference to Test First Programming or Test Driven Development? If so then your statement would be accurate if it said "Code/test is a method to design code at the implementation level."




Sounds like, smells like... still does not make something true. Oh well, am I Maverick and live on the frontier of Agile and every other methodology. I do not make any money from their books, I am not an Agile consultant, and so my interest in Agile is purely from an "application" point of view. I still have the challenge for the authors of the Agile methods you question to precisely define what it is their method does, how it does it, and what is to be expected from doing it. I think they would argue that the current body of knowledge has defined their processes and the answers can be found in their books, papers, and statements. For instance, who owns XP and thus IMO is the only one that can say what XP really is? In my opinion it is Kent Beck and Kent Beck only. Who owns Crystal Clear? Alistair Cockburn. Who owns Maverick? I do! Any one else's statements are mere opinion in my opinion.




I am glad you have read some of my posts. Thank you.




As for what is needed for code, what I am describing is the simplest set of "things" needed to write code. What you are concerned with is developing software that meets the needs of a specific customer. That is something different all together. What do you need to meet your concerns? I will not try to address that here but you need a lot more than a computer and something or someone to generate instructions for the computer to execute!




I have clients that see giant leaps in productivity, quality and profitability when they learn & study the core business and do not wait to be told what to do.




First, you have declared that Agile is Code and Test which is Trial and Error wich is sit and wait to be told what to do next. I still say that this argument is invalid.




Second, I have never worked anywhere where the programmers sit and wait to be told what to do next. Never in all my years. You paint a picture that is quite distorted to me.




Here is a straight up question.


Have you ever been at an XP development shop where the developers sat and waited as you describe and then you introduced your method and experienced the leap in productivity that you describe?




Have you ever participated in gathering requirements as defined and described by Alistair Cockburn? How does Alistair's method of Use Cases ignore the customer and create a "software waiter" situation?




As for my posts on Digerati Illuminatus (tech.groups.yahoo.com/group/digerati-illuminatus/) you criticize my postings because I do not take the business perspective. Well sorry but I have to say, DUH! That wasn't the point. I will do this, propose some topics and I will work with you and write a blog posting including the business perspective and post it to my site and I will be your co-author on the paper.




It is not up to me to prove Agile wrong. It is up to Agilistas to prove it right.




Well, you started this with the fervor as if to say you are protecting all businesses that depend on software from the latest scam called Agile. If your critique is to be taken serious then it should be of the highest professional quality. You state the paper was not written as an academic paper. So what is it written as? A statement of facts? An attack? An appeal? What?




Here is my battle cry for IT, LEARN THE BUSINESS! If you don't learn the business, you cannot participate in strategic planning and especially release management.




Is this scaleable? I imagine you are describing a situation where a consultant that comes into busineses with very focused and specific needs which results in a small team of 1 - 4 developers that are hired to "attack" the problems. The companies I have worked at do not want ALL of the developers participating in strategic planning and release management. There may be hundreds of programmers at the company. There are near one hundred where I work now!. So are your recommendations wholesale usable by me? No. So what types of business, what size of business can use and sustain having all of development doing strategic planning?




David, you admit you didn't even know what many of the logical fallacies are and now you have a recommended reading list! WOW! I am not certain you are an authority on logical argument. I am certain you are not an authority on Agile software development, but you are learning and reading, and thinking about it. Try a little application and experimentation, yes I know this is emperical. You do have first hand experience with the Agile practices, right?




David, I appreciate your comments and your concerns.




For the record, I do not make any money from Agile Software Development. I have no books on the topic. I do not consultant on the topic. I do not travel and sell it. You really need to get a discussion with the authors and I am not one of them. I have critiqued your arguments and I find them unsound. That's all. If you present accurate representations of Agile practices I feel your arguments would be "better".




David, I am sincere in that I have nothing against you. If there are any personal attacks in my post I appologize. Even my arguments fall victim to common fallacies.




Thanks,




Geoff




P.S.


You sure like P.S.'s alot!




P.S.S.


Geoff try not to take this argument or yourself too serious.




Okay. I am not a serious person on a personal level. If the paper wasn't meant to be taken seriously then first, sorry for the time I have taken from you, the community, and even myself.




If the paper isn't serious then I feel it was just a scheme to attract attention to your website and business and thus I have been victim of an adverstising ploy. A little controversy goes a long way.




Consider the topic closed and on the "not to be taken seriously" list.

It's amazing what a simple google search of "agile longstreet" will turn up :>



For the record I guess I am an agilista, even though I think the term misrepresents the spirit of Agile. I think of myself as more of a pragmatist and not particularly dogmatic (but I was raised as a Catholic, so I might be wrong).



I've been using agile for several years, both successfully and unsuccessfully. On the unsuccessful projects ,always the most interesting, the problems have been with a flawed implementation of the concepts, not the core concepts. On the plus side, Agile's insistence on constant reflection and improvement, IF PRACTICED, can provide a mechanism to correct problems.



After reading David Longstreet's article several times, and both supporting and critical reviews of it, I'm left to conclude that Mr. Longstreet just isn't interested in learning what Agile really is. I'm not sure why, but anyone that spends some time to learn what agile is all about will recognize the fallacy of his arguments. Sure agile isn't a "silver bullet" ,to quote Fred Brooks, but it can be very helpful in many circumstances. Unfortunately, many will take the easy way out and accept his criticisms as fact. Me, I like options, or to put it another way, more tools. Agile techniques provide more tools for me to put in my bag, more options to have available when tackling those "opportunities disguised as unsolvable problems".



This is all a long-winded, and admittedly not a particularly agile, introduction to another discussion that came up on the google search I mentioned at the beginning of my post - www.targetprocess.com/blog/2007/06/erroneous-an.... I found it interesting, maybe others will too (but then again I'm a commie pinko agilista :>).



Finally, I think Mr. Longstreet does endorse the concept of "decide for yourself". By all means do read his article. He does a good job of raising arguments that come up again and again against agile techniques. But before you decide, do some homework on what leading advocates of agile and lean techniques say too (Mary & Tom Poppendick, Jean Tabaka, Mike Cohen, Martin Fowler, Kent Beck, Alistair Cockburn, Hubert Smits). There are lots of good ideas out there. Pick the ones that make sense to you, try them, but by all means continually evaluate whether they're working for you.



Cheers,


Rich



P.S. Peace!