Requirements Hubris: A human condition

What is it about requirements that make us sure that we can get them all and get them exactly right at the beginning of a project? I’m not just talking about software or digital products, this phenomenon (human flaw?) seems to exist everywhere including, but certainly not limited to: home improvement, instructional design and event planning projects.

Take my latest Agile process improvement project (the irony of this will soon become apparent). I was certain I understood exactly the outcome the client wanted and even believed I had some solid clues of how they might measure a successful outcome. Yet there I was at the end of the project answering a question that sounded something like: “but how come we can’t do this (or that) now?”. Didn’t I elicit all those expectations in my success criteria brainstorming session at the start of the project? Just goes to show, even if you do the right thing (in my case a success criteria brainstorm) it does not guarantee that you will get it right.

You will have to excuse me for sounding like an Agilist with what is about to follow. I always fancied myself an agnostic when it came to project methodology. I use what fits the situation and my clients’ culture best. Maybe what fits best for getting the requirements and getting them right is to progressively elaborate them. Maybe there really is no other choice when the artifact of a project is abstract until built and resists both simulation and facsimile. That’s certainly the case with a software or digital product as well as the 3 other examples I listed above.

Why should we expect it to be any different when it resonates as true that we don’t know what we truly want, indeed we may not even know we wanted it, until we see and use it. A meal on a menu may sound good but we won’t know whether we like it until we try it. A tropical vacation spot is described to us by a trusted source but we leave our vacation disappointed. These are examples of unmet expectations and requirements. With software solutions, it is frequently the case that we force our users to adopt our interpretation due to the expiration of time or the lack of money (click above picture to see the “elephant cat” example).

We really shouldn’t be surprised by the scenario where we elicit a long list of expectations stated by the future user of a software system/product then document an interpretation (albeit an unintentional one) of what we heard only to discover, when the built solution is delivered to the same future user, that it is not what they wanted. According to The Standish Group, almost 65% of all developed system features are never used (see pie chart). The only explanation for this is that the original requirement was never accurately heard, documented, interpreted, or some combination thereof.

So what is the solution to the hubris of this “I can get it all up front” human condition? It’s progressive elaboration. We know that we don’t know what we want until we see it, so why not start by making visible what we think the solution is to those who will eventually use it? Software solutions lend themselves very neatly to progressive elaboration. A home renovator would relish the opportunity to partially build a working solution without having to tear it down and rebuild if/when their interpretation of the requirements are discovered to be flawed, but that is just not practical or economical. A restaurant chef can’t provide incremental tastes of a dish until it exactly satisfies your requirements. Only with software can we, with relative ease, build the first small increment of a solution, present it to the future user of the solution, have them use it and modify based on their feedback. This process can then be repeated until the solution is exactly what the future user of it requires. No more and no less. We are in an advantageous position in software solution development yet we far too frequently overlook our advantage and resort to the aforementioned hubris.

Maybe that’s why the word progress is embedded in progressive elaboration. It is the way forward from the “I can get it all up front” human condition to a better state. As Peter Drucker said, “there is nothing so useless as doing efficiently that which should should not be done at all”. It’s easier then we think to begin to stop delivering software solutions that should not have been developed at all.