Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal

Thursday, August 28, 2008

The Silver Bullet

As I just read in a comment about silver bullets, I couldn't resist to address this topic :-)
Frederick F. Brooks brought up the concept of silver bullets. Of course, Frederick was not refering to some weird persons who are turning into computer geeks during full moon and can only be stopped using these nice little silver bullets. Instead, he claimed that there never ever will be any software technology that can boost software development productivity by a factor of 10 or more. So far, he has been right. Object-Orientation? Really productivity-boosting, but not to an order of magnitude. Models? Nice try, but someone has to build the model and the generator.
But won't there be any technology in the future that can refute this thesis? Suppose, an alien lifeform with incredible technology is visiting planet earth. How would they design software systems? Do we really believe, they would leave their giant space ship with a notebook, start modeling their systems with a kind of UML tool, and program in Java?
Thus, could the Silver Bullet be a universal law, or just a kind of constraint only humans are facing due to their limited capabilities?
Frederick F. Brooks also brought up the concept of accidental versus inherent complexity. Basically, each problem has an inherent complexity. You can't specify a a linear algorithm for the general sorting problem. Forget it, there is really no way! But we ourselves are causing a lot of problems by accidental complexity. Using an array for storing a sparsely populated 1000x1000 matrix isn't a very smart approach, right?
No technology can get rid of inherent complexity, but it could hide this kind of complexity from the user. Hiding complexity is one way to increase productivity - just make it a SEP (Somebody Else's Problem). Helping to address accidental complexity is another. Consequently, the best productivity boost results from combining both.
According to philosopher Thomas Kuhn, evolution of technologies is more a linear process. A technology can only scale upon specific limits. If it reaches the limits, workarounds will help postponing the necessity for a new approach. But at some point in time, a completely new and revolutionary approach will appear (such as switching from structured programming to OOP) and the same kind of technology cycle will restart. It is a kind of spiral model if you will.
The only point where we could observe productivity boosts are the technology revolution events. However, new technology revolutions are mostly rooted in existing technologies. Thus, an immediate productivity gain is simply not possible.
But as a thought experiment: if you just compared the way of approaching software development challenges now and 40 years ago, you could recognize a tremendous increase of effectiveness. There is a long way from software engineering in the sixties to that of today. No person could have skipped all the intermediate results and challenges.
Thus, if an alien race would land on earth, it appears very likely they could introduce us to new ways of building software systems that offers magnitude of productivity increase.
Conclusion: Silver bullets are theoretical beasts only existing in the Star Trek Universe. As pragmatic architects we must leverage the tools and technologies existing today. And we shouldn't believe all those technology panaceas vendors and market analysts are constantly promoting such as SOA, SaaS, you name it.

Tuesday, August 12, 2008

Now it is proven - NESSI exists!

ok, but what is it?

According to its web site, "NESSI is the European Technology Platform dedicated to Software and Services. Its name stands for the Networked European Software and Services Initiative."

Yes, it is not very clear from this definition. So, let me try my own explanation.

Suppose, you'd like to leverage SOA to connect heterogeneous islands. Maybe, you are going to build a healthcare system that connects different international regions, allowing physicians to access your healthcare data if required and permitted. An underlying SOA infrastructure should provide:

  • SOA based middleware and process modeling
  • Enterprise Application Integration facilities
  • a kind of internet bus that supports various protocol families
  • registries and repositories
  • horizontal services such as security
  •  

Sure, you can obtain such ingredients, if you accept vendor lock--in. I don't list all the big SOA proponents here as they are quite obvious. The truth is that is almost impossible to cope with such heterogeneity. If you try, you will be soon overwhelmed by sheer complexity.

Unfortunately, there is another dimension to this problem:

  • Even if you have all these facilities at hand, you need to standardize the vertical domains. What if the whole SOI (Service Oriented Infrastructure) is standardized but two banks don't agree in what a customer or bank account is?

So far, solutions have either attempted to address the technology issue or to address the domain standardization.

What NESSI tries is to address those challenges.

It is required to create a kind of standardized SOA operating system and to define domain-specific standards for how to create applications on top of this platform. From my personal viewpoint the combination of Open Source Software with standards would be a perfect fit. It is necessary to create both, standardized APIs and a reference implementation able to handle mission critical applications.

Is a platform sufficient? Absolutely not, because we also need to cope with service engineering and governance issues and non functional requirements and tools. Consequently, NESSI is also trying to address these points.

All of these aforementioned  problems are hard to solve but on the other hand we have to solve them anyway, because they root in inherent complexity. If we need to solve them to finally live the dream of "the network is the computer", then NESSI represents an excellent opportunity.

But that's my personal viewpoint. What is your's?

Michael's Event Calendar

 

  • In September I will visit my colleagues in Bangalore and give a seminar on Distributed System Architecture.
  • On 14th October I will give a keynote on COMPARCH 2008 in Karlsruhe. I was invited by Ralf Reussner and Clemens Szyperski, two well-known gurus on architecture and components. Guess, what topic I will cover? Yes, architecture refactoring, my preferred topic these days. 
  • Originally, I planned to attend this year's OOPSLA conference in Nashville. I have two accepted tutorials, one on high quality software architecture, the other one on Software Architecture Refactoring. Unfortunately, I am so busy within Siemens that I won't make it to the OOPSLA. However, an excellent member of my staff will give the two tutorials instead. Marwan is not an emergency solution but an excellent architect himself, formerly working for Lufthansa Systems. You will have a lot of fun attending our two tutorials. Mentally, I will be there, of course :-)
  • In November I will most likely be invited to the Microsoft TechED in Barcelona giving two talks.
  • In December I will be available on a panel at the Service Wave Conference of the European NESSI initiative.
  • In January 2009 there is my local event, the OOP 2009 in Munich. I don't yet know what topics I will address but be sure I will be around :-)
  • I am planing to submit for next year's ACCU in Oxford (April).

Mojave - Episode II

Did you hear about Microsoft's Mojave Project where they told a bunch of selected people - most of them Vista critics -  of a cool new operating system called Mojave? Supposedly, most of them were kind of enthusiastic about Mojave. To their big surprise, Mojave turned out to be Vista - the devil in disguise if you will in remembrance of an old Elvis hit.

What can architects learn from this?

  • First of all, we can hide all the mess of our systems behind a nice look and feel. The best software system will not be accepted if its usability sucks, and vice-versa.
  • Secondly, if we got an excellent system that people for some strange reasons don't appreciate, we could try to put a new API or Look & Feel on top. Maybe, the interface is the only thing that sucks in our system.
  • Last but not least, we could also learn that nice architecture and design diagrams might impress others so much that they consider the system as a neat piece of design while in fact the implementation is totally different from the design.

Regarding the original Mojave experiment I really doubt that the reviewers had the write level of expertise and mind set. Don't judge a software system by its cover!  Review and evaluate the design and the implementation thoroughly. 

Architects shouldn't just believe but rather evaluate and enforce.

But of course, in some stakeholder presentations such as presenting the system to your top management you can add a marketing layer while remaining politically correct and honest.  

Monday, August 11, 2008

Art, Craft or Science?

Is software design an art, a craft or a science? This question has been the starting point for many entertaining discussions. And many different opinions and answers exist.

So what is my take on this?

To build a software system requires engineers to come up with a mapping that takes the problem and maps it to a concrete solution. Requirements describe the problem and the desired properties of the solution. Unfortunately, the mapping typically is not unique in that there might be many solutions solving the same problem. With other words, architects and developers face a lot of freedom. The bad thing about freedom is that it always forces architects to take decisions. In other words, to architect is to decide.

Of course, we appreciate if we can immediately find an appropriate solution for all problems, just following a divide-and-conquer strategy of software development.  But this never happens in practice.

Two problems come to my mind in this context:

  • Sometimes we encounter problems for which we neither know a solution nor can find a solution elsewhere (finding a solution means either knowing a direct solution or being able to map the actual problem to another problem for which you already know a solution - for example, if we are able to map our algorithmic problem to a sorting problem.)
  • In contrast to mathematics, the problem itself is a moving target in software engineering. Pantha rhei.

The first issue implies that you may not find a pattern or some other kind of best practice for your concrete problem context. Instead, you need some intuition. Hopefully, you recognize that the problem at least resembles another kind of problem for which you already have a vague solution idea so that this can serve as a good starting point. By the way, this is the reason why it turns out to be so hard to build a general problem solver for software engineering. Freedom is very hard to deal with by a general problem solver.

The second issue, coping with change, requires some sort of agility. In contrast to logic, our axioms keep changing while we are applying the calculus. The only way to deal with change is to embrace change everywhere, in the process, in the tool chain, in designing, in testing, in refactoring, in programming, in communicating.

What makes software design so hard sometimes is when both problems appear in parallel.

When people ask whether software engineering is a craft, a science, or an art, the answer should be obvious now. It is a science as we try hard to follow formal approaches where feasible. It is a craft because solving problems often requires experience and best practices. And it is an art as software engineering leaves enough freedom for intuitive and even artistic solutions.

I guess this is what makes our job so interesting. 

Friday, August 08, 2008

Architect always implements

When I once invited Bertrand Meyer to our location in Munich, he gave an excellent talk on Software Engineering. His most memorable messages were "Bubbles don't crash" and "All you need is code".

With the first message "Bubbles don't crash" Bertrand referred to all those engineers who have turned into Powerpoint or UML freaks. A Powerpoint slide or UML diagram will never crash (if you don't use it as a model for code generation, the MDSD guys might argue). It is surprisingly easy to impress people with simple boxes and lines (sometimes dubbed software architecture). Especially it is rather simple to impress top management which is why the wrong consultants can potentially cause so much damage. Some might respond that high abstraction levels are the only way to understand and build complex systems. That's certainly true. On the other hand, you would not be able to build a ship if you never had any idea about water. The same holds for software systems. It is simply not possible to develop a software system without any idea on top of which runtime environment it is supposed to execute.

Even if you had in-depth knowledge of your software system's runtime environment and built an excellent software architecture, it is NOT a no-brainer to transform a software architecture to a working implementation. That's why Bertrand introduced the "All you need is code" attitude. What helps you the best theory, if you can't turn it into practice? As an architect you need to have implementation skills, either for implementing parts of the system yourself or for assessing the results of other team members. How could you ever build a good bicycle without biking yourself?

The problem arises how to ensure that we as software architects can meet these requirements. "Architect always implements" means that you should also implement in a project, but definitely not on a critical path. By implementing you will earn the respect of developers, while keeping in touch with the implementation at the same time. And you will have to eat your own dog food. Whenever something is not as expected you can detect the problem and help solving it. However, in some projects architects are busy due to communication, design, and organization efforts so that spending time for implementation will be almost impossible. Never say impossible in this context, do not even think it!  Even in such scenarios I recommend spending some of your time for code reviews and some of your spare time to get familiar with software technologies being used in your project context. Otherwise you will create kinds of Software Frankensteins: creatures you are not proud of and who tend to live forever unless not eliminated in an early stage.

Architects are not just masters of Microsoft Office and  of a bunch of drawing tools they leverage to create design pearls, throw them over the fence to the developers who ought to build a perfect implementation, while the architects enjoy their holidays on a nice Carribean island.

Be pragmatic! I guess, the Pragmatic Programmers book series was created for exactly the same reason; to make us aware of the relevance of practical experience.

Never ever forget: ARCHITECTS ALWAYS IMPLEMENT. There is no exception to this rule.