Monday, April 23, 2007

On Apples and Oranges

In almost all discussions about the maturity of software engineering, people refer to other disciplines claiming that software engineering is relatively immature. One of the arguments popping up all the time addresses the inability of software engineers to systematically map requirements to specifications. In addition, software components and their composition rather resemble primitive tools such as the ones used in the stone age according to those critical voices. Software enginering happens in an ad-hoc manner while other disciplines have learned how to apply a scientific formalism or a systematic engineering approach. What a shame to be a software developer. A point often made is the industrialization experienced in other domains such as hardware manufacturing where re-use is the norm and not the exception. It is my basic claim that most of these comparisons are more of the apples and oranges kind of style. So let me state some thoughts:
  • While it is certainly true that industrial hardware production happens with a huge amount of re-use of standardized components, software engineering is more like hardware engineering. At design time hardware developers face exactly the same kind of problems when identifying requirements and mapping these to concrete circuit plans. This is the same mixture of craft, art and science software engineers rely on.
  • Software systems experience a lot of more changes during design and implementation than their non-software cousins. Would you ever expect that shortly before a Boeing 767 is ready to be delivered someone will ask the engineers to change the body or engines. But that is exactly what happens in software engineering. Changing an airplane engine during flight? Impossible for aircraft engineers but expected from software systems. Add to this that customer requirements are much more unspecific, dynamic and unreliable in software development projects.
  • Software must run in a lot of different contexts. Take operating systems such as Mac OS X, Linux, Vista as an example. They must support different kinds of hardware systems, different kinds of drivers, different kinds of user preferences. Such flexibility is not provided by any other product. For the same reason, large-scale software re-use such as the component or service market places we often dream of will remain dreams despite of standardization.
  • Some people tend to make humorous comments about software engineering and rocket science. The problem is that in terms of complexity many software developent projects reveal a large degree of complexity. Software engineering sometimes IS like rocket science. Costomers and developers often ignore this complexity which is caused by operational and developmental qualities. The largest software projects are way larger than any comparable non-software projects. Keep in mind that the same holds for all the software tools and infrastructures.
  • An incredible amount of bugs lurk in software systems due to a lack of quality in software developemt. True or false? Have you ever seen the error list of hardware systems such as that of new CPUs? Believe it or not, but I claim that the error rate in software systems is not higher - at least not significantly - than the one in comparable non-software systems. In addition, we should keep in mind testing is much more difficult in software due to the higher diversity of the software ecosystem.

I propose the following experiment: take a bunch of hardware developers who have become software engineers and see whether they are more productive and offer higher quality when developing software than their colleages without that background. From my viewpoint the immaturity of software engineering is a myth. Wrong statements repeated often will not become correct statements. I am wondering what other people think about this.

3 comments:

Anonymous said...

I do believe with proper planning and software design most of the bugs and bottlenecks can be eliminated at the planning stage itself

mixi said...

I want to mention two things. First of all, you've got a glorious sentence in your text: "This is the same mixture of craft, art and science software engineers rely on." Notable that you talk about an engineering discipline and say that it has some science, some art and some craft but nothing to do with engineering. That's curious, but I almost agree on that. Sometimes I even think it mostly consits of art, but anyway - let me state my second point.
When I think about software engineering and the act of creating a piece of software, that one sentence, Albert Einstein once said, comes repeatingly to my mind: Things should be made as simple as possible, but not any simpler. What this means for me is, that when you do something "any simpler" you probably oversee something and one day in the future you will get into trouble. I give a practical example: I decide to build a software application using some scripting language. As the software grows, the choice reaches it's limits (like performance, scalability and all the usual buzzwords). When I would have started that project using a proven software building environment (like C++), I would have done a better choice. So what's the conclusion? I've done something simpler instead of simple. You get me?

Michael said...

That's a very good point, indeed. Your example of a software system reaching its limits is also perfect. There is a good book (Henry Petroski: To Engineer is Human) which demonstrates that in all disciplines further evolution is triggered by old technologies reaching their limits. Take the space shuttle accidents or instable bridges as examples. We need to learn from failure to reach new limits. And as the book illustrates, this is a general engineering problem, not a software engineering specific one. BTW, your example is more an example of an simplistic approach. In such a concrete case I would ask the project team about the concrete problem context and requirements they had in mind. Maybe, the stakeholder who ordered that product has specified insufficient or wrong requirements. Or maybe, the development team simply did not map requirements to the implementation. Anyway, this is nothing which is constrained to software engineering.