Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal

Friday, November 07, 2008

Why the waterfall model does not work

Again and again I have met managers and software engineers who tried to convince me that agile processes while being appropriate for somebody else's project were simply not applicable to their problem contexts for several reasons. They came up with rather lame reasons like being used to waterfall models or the claim that agile processes do only work for small teams. Especially I am enjoying people's reaction when I am talking about really huge projects I had been involved and which followed the Scrum method.

From my viewpoint the question shouldn't be whether to use agile development processes instead of a waterfall approach. I would rather suggest to let all those waterfall addicted prove why they consider waterfall approaches more suitable in terms of a concrete project. 

It is true that an agile method is not superior in planning projects. Even in an agile context you will need to estimate resources, budgets, or time-frames in advance.  But in contrast to a waterfall approach you get an early feedback if your initial planing proves to be plain wrong: If after the third iteration you got an substantial delay you are able to react with appropriate countermeasures. In a waterfall model the old saying is that while 80% of the project have been completed, you still need to address the remaining 80%. You may be doomed to fail at the beginning of the project but a waterfall model lets you live with this illusion till the sad end. An agile method on the other hand just provides a whole range of safety nets. You may fall but contact with ground usually is much smoother.

Let me take it the other way round. What are the preconditions to make a waterfall model succeed?

  • All requirements are available at project start and will remain unchanged.
  • All of the requirements are consistent and complete.
  • No errors will be introduced in any phase of software development.
  • Likewise, all tools, legacy components, infrastructure parts fully adhere to their specifications and, of course, are consistent and complete themselves.
  • All stakeholders start with a full understanding of the problem domain. Developers, testers, architects also have a full understanding of the solution domain.
  • There will be no misunderstandings in any communication between stakeholders.

Obviously, most of these preconditions never hold in any real-life project. We have to cope with error and failure. In agile approaches, the principle of piecemeal growth helps to master complexity. Safety nets such as test-driven development or reviews enable quick detection of potential problems. Means like refactoring support us in addressing the problems we detect and to embrace change.  Agile communication and customer participation  help in keeping all stakeholders synchronized.

Two of the main principles of agile engineering might be called "embrace change" and "learning from failure".

And if we look at nature, evolution basically also represents an agile approach. For all creationists: evolution does not necessarily rule out that there could be a master plan behind everything.

 

What's your take on the agile versus waterfall dispute?

4 Comments:

  • I agree more or less BUT I believe that the waterfall model and especially "waterfall-model-estimation" is often the only way to get a contract in the first place.

    If the customer does not see benefit of an agile approach your options are limited. Of course you could try to convince him that in the long run the chances of the project to succeed are higher and he will be more satisfied with the software but in the end: he is the customer and pays the bills.

    Have you read the interview with Ernst Denert in the latest Objektspektrum? http://www.sigs-datacom.de/sd/publications/pub_article_show.htm?&AID=2440&Table=sd_article

    He dismisses XP as step in the wrong direction and uses the "customer-pays" argument IIRC. It is a very interessting read.

    By Blogger valentin, at 11:07 AM  

  • Hi Michael,
    I think that as agile methodologies are versatile to adapt projects life cicle, we must also be versatile enough to adapt to agile methodologies.
    With this, I mean that you don't need to use all the practices defined for a methodology to the letter, but you could modify them (or avoid using them) to satisfy your needs and.
    I think the main reason why many people reject using agile methodologies is because they're not mature enough in SDLC to define the practices that will make the project work out.
    Just MHO.
    Dan

    By Blogger Daniel Zuazaga, at 12:43 AM  

  • I think a pure waterfall model is dead. Perhaps it worked when business was not moving at the "speed of light" in earlier days. In this day and age, when even traditional industries experience hypercompetition and product lifelcycles are so much shorter, the concept of waterfall is outdated to say the least.

    By Anonymous Anonymous, at 1:41 PM  

  • Most of the systems around you have been produced by a watefall-like approach.

    The preconditions are simply untrue.

    It is the very purpose of the analysis to determine all the requirements and the specifications. Trying to obtain all of them is an illusion. It would lead to analysis paralysis. Thus having a fairly good and stable picture of the product to be built is right.
    The project continues with a, let's say, for 80% or 90% stable picture of the product to be built. The changes that may still occur shouldn't be about the core of the solution anymore and should have limited impact. Of course, nobody is ever fully protected from fundamental changes. Never. On no project whatever the methodology is used. Even scope changes are possible. So, modern "waterfalls" are not written in stone.

    The gathered requirements should indeed be consistent. If a requirement changes, consistency must be re-obtained. It's the same with code, isn't it? All the programming code should be consistent.

    Errors may happen at any time. That's why there is a lot of verification all the time in waterfall projects: frequent reviews with users, validation, V-or W-testing model, ...

    The insight in the problem and in the solution evolves. That's the very purpose of analysis.

    Why should there be no misunderstandings? Of course there are. This is why frequent communication is important and why frequent reviews are needed. Waterfall doesn't prevent this. It is perfectly integrated in this process.

    By Blogger Axel Vanhooren, at 3:30 PM  

Post a Comment

<< Home