Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal

Friday, March 31, 2006

What should be in an Architecture Document?

Recently, I was asked by a colleague how an architecture document should look like? What are the basic rules each document should follow and how should all documents be related to each other? Obviously, there is no standard template applicable to each project context. However, there are some attributes a really good architecture document should reveal:

  • Introduce special documents on the domain model (if applicable), on central issues such as patterns used, and on different kinds of guidelines. Make sure all architecture documents comply with these guidelines.
  • Don't extend 50 pages per document. A large document with more pages will not be read by anyone, at least not all its content.
  • This is a no-brainer. But nonetheless, I am going to mention it: Use a standard document template.
  • Structure all documents exactly the same way. Otherwise, readers won't feel comfortable, especially when they have to read more than one architecture document.
  • Describe the software architecture top-down. Start with the basic abstractions and then step by step move to more concrete details and implementation issues. This way, different readers can read the document according to the detail level they need. For instance, a manager might only be interested in the coarse grained architecture while engineers are also interested in deployment issues. A good way is to use the 4+1 view of the IBM Rational Unified Process. In the beginning, explain what part of the architecture you are actually describing in the document including its motivation and scope. Then address all requirements that have influenced your architecture (decisions) as well as the relevant use cases that drive the architecture design. A context diagram helps to show boundaries of your architecture as well as its integration into the rest of the system. Introduce the logical view with CRC cards, class diagrams, sequence diagrams. Don't come up with technology-centric issues too early. Even if you are using .NET Remoting and C# and SQL Server in your project, to name an example, your architecture baseline should not depend on these technologies, at least not its logical structure. Complete documentation using step-by-step refinement up to the deployment view.
  • To ensure symmetry, always use the same diagrams for the same purpose such as UML diagrams, ER diagrams, or domain-specific languages (DSLs). Also make sure, that these diagrams use the same meta-model. For example, introduce project- or domain-specific stereotypes in a separate document (see first bullet).
  • Add a glossary that explains all relevant terms and abbreviations. And make sure, all documents use the same terms with the same meanings.
  • Add references to other documents that are relevant for reader understanding.
  • Motivate all important design decisions that are not obvious. I've read so many documents that introduce architecture without presenting the rationale. Another software architect that reads your document must be able to understand the rationale (the "why" not just the "what"). Understanding the rationale also means understanding the architecture.
  • For the same reason, always explain in your document how your architecture addresses the requirements. Thus, requirement traceability will be possible.
  • Use patterns and explain the patterns used in the architecture document without providing full pattern descriptions. Rather refer to existing pattern descriptions in literature or special pattern documents.
  • Structure the architecture documents in a tree-like hierarchy. The root is defined by the overall strategic architecture document that introduces the whole system. In the root document introduce all relevant subsystems as well as cross-cutting concerns such as security, scalability.
  • The nodes right under the root then denote subsystem documents and further documents that address the most important cross-cutting concerns. If your subsystems consist of large entities themselves you may add additional leafs to the document hierarchy.
  • Also specify testability (such as unit tests) in your document. This defines the specification of your architecture as well as how its implementation can be tested.
  • Check-in all your documents in a configuration management system. Use versioning for your documents.
  • Write the lead author or all co-authors onto the document cover page, so that a reader might track down the authors. Also specify date and version.
  • Let other people review your document (and design). Feedback should be related to comprehensibility and technical soundness. Don't rely on the colleagues that are in the same subteam but also ask other people (managers, architects, testers) to read and review your documentation.
  • A more process-related issue: make sure your architecture document always reflects the current state of the implementation (if already available). Otherwise, reading your document might be a complete waste of time and even cause other teams to draw wrong conclusions.
  • Add diagrams and figures to explain your design, especially for complex architectural issues. No one will be capable of understanding hundreds of lines without any visualization. Drawing diagrams requires additional efforts but is always worth the time being spent.

These are some recommendations I have derived from my work as software architect. I am sure there is even more. If you find this helpful or if you have additional suggestions or feedback don't hesitate to add your comment.

Thursday, March 30, 2006

Middleware - what Middleware?

In many solution architectures communication middleware serves as the central backbone which helps to glue all core parts together. It is obvious that the selection of the appropriate middleware is a critical issue in any development project. Unfortunately, in many projects this decision is driven by political forces or personal preferences. Managers often do not recognize that the selection of inappropriate infrastructure technologies or tools can be disastrous. Even if architects are ready to base this decision solely on use cases and requirements, it can be incredibly difficult to move to the right direction. As Andrew Tanenbaum once said with respect to standards "there are so many that it is difficult to choose from". So, the first question is: what kind of middleware is appropriate for what kind of problem? To answer that question it is important to get an overview of middleware types and paradigms.

Basically, the following kinds of middleware exist:

  • Messaging Middleware: applications send messages to each other. Message contents are application-specific while the structure and header information is specified by the MOM (Message-Oriented Middleware). Messages might be sent from one peer to exactly one other peer (end-to-end or queue-based messaging) or (anonymously) from multiple publishers to multiple subscribers (publisher subscriber or topic-based messaging). Examples: MSMQ, MQSeries, JMS, SonicMQ, JBossMQ.
  • Remoting Middleware: hides all communication details from developers by extending conventional operation calls over the network. Clients and servers can almost be implemented as if they were residing in the same address space. All communication issues are handled behind the scenes by glue components that are typically generated using tools. Examples: RMI, WCF aka Indigo, CORBA, ICE, DCOM.
  • Eventing Middleware: focuses on distributing fine grained events. Very similar to messaging middleware and thus I won't cover it here in detail.
  • Distributed Transaction Monitors: provide transactions management across different components. Examples: Tuxedo, CICS, MTS. Will not be covered in this posting as transaction monitors today are rather integrated into the other types of middleware.
  • Service-oriented Middleware is a kind of meta-middleware used to integrate other middleware. Most prominent implementation are XML Web services. Web services are mostly useful in business-level integration scenarios.
  • Peer-to-Peer Middleware: a combination of Messaging middleware and Eventing middleware where the locating of remote peers happens through discovery algorithms instead of relying on centralized repositories.
  • I won't explain Multi-Agent-Systems here as they are only used rarely.
  • Neither I will explain EAI systems here as they are basically built on top of the aforementioned middleware types.

Therefore, the first distinction should be whether your problem domain requires a more method-invocation based or more message-based approach. Asynchronous operation is one of the important issues here. Despite of the fact that some remoting middleware technologies have also introduced asynchronous method invocations (e.g., CORBA), messaging is more appropriate for asynchronous communication. Asynchronous communication basically means, that the sender for its subsequent processing does either not expect a result from its communication peer or does not need the result now. Hence, senders and receivers should be decoupled. Example: sending a purchase order to the order processing subsystem. Another issue for the preferred usage of messaging is when you don't want tight coupling between communication partners. For example, a server that is sending notifications to different receivers which are interested to subscribe for obtaining different messages from different senders. A further example is the provisioning of advanced communication styles such as broadcasting or single request - multiple replies. These styles do not map to existing programming languages, anyway. Thus, it makes no sense to provide them through remoting middleware. Last but not least, scenarios where communication links are not reliable are more appropriate for messaging. Take a mobile phone as an example that lost connection with the network.

If, however, the sender needs an "immediate" result from a specific communication partner, remoting is much more suitable. Example: asking a credit card company for validation of a specific card in a Point-of-Sale system. The advantage of remoting middleware is the transparency they provide. Basically, developers can ignore all communication details. But that is exactly the problem. Developers often tend to leverage remoting middleware as if they were developing non-distributed systems without taking issues such as latency into account. You can easily imagine the performance of such systems. Another problem with transparency is hiding of details can be a disadvantage in terms of tracking errors. Because everything is hidden, error causes also are hidden. In summary, usage of remoting middleware and transparency are totally overestimated in practice.

Messaging middleware can be extended to provide remoting. There is a simple reason for this. Every remote invocation can be separated into two message transfers. A request message is transferred to a receiver which then sends back a result message to the originator. Actually, all remoting middleware is built upon some kind of messaging infrastructure. In contrast to some claims, the opposite is not true as asynchrony can not be guaranteed when remoting middleware is used as the underlying base for a messaging layer.

Peer-to-Peer middleware simply combines messaging and eventing with discovery strategies. Instead of looking up a receiver's location from a known repository, it is much more reliable to discover a resource, especially when the same resource is available more than once in the network. In a Peer-to-Peer system a consumer or sender does not care about which concrete provider or receiver it is using. Thus, Peer-to-Peer approaches are particularly helpful for coping with decentralization. In theory, a similar approach is possible for remoting middleware. Instead of asking a repository where the server object is located, it is possible to use a trading service, ask for a specific service type with specific properties, and get possibly multiple object references returned that meet the specified type and property constraints. This is the right way if you need a remoting based approach combined with decentralized lookup strategies.

SOA basically means loose coupling. For example, peers are implementation-agnostic. The only thing an application sees is the service interface of the application it need to communicate with. The communication itself uses a commonly agreed messaging protocol. This approach can be best implemented using XML Web services and Messaging middleware. SOA is particularly useful to integrate different middleware and application islands with each other.

This posting could only scratch on the surface. There are a lot of issues that could have been discussed such as error handling, fault-tolerance, scalability, security. My intention was to illustrate that no middleware paradigm can be a general solution for all problems. Thus, it turns out to be essential to first specify the problem space and then identify which middleware solution is the appropriate one. For the same reason, it is sometimes not possible to rely on one single middleware technology because there are multiple problems requiring multiple middleware solutions. Remember: select your middleware carefully and base this selection only on the problem domain, not on political or personal preferences.

Making Noise

Yesterday, I addressed the issue of meeting overkill in this blog. This was a first example how productivity can decrease in any software engineering project. Another point of trouble from my viewpoint is e-mail. Do you remember those days when an incoming e-mail was a seldom and happy event? Neither regular spam nor internal spam? Sure, that time the number of e-mail accounts was significantly lower. On the other site, I remember that more people followed the netiquette. Today, in a regular project, especially when it happens to be multi-site, there are tons of e-mails daily (many of them related to unnecessary meetings :-). Add to this company-internal mails and all those spam mails and marketing mails and mail distribution list mails, you name it. How would you ever be able to handle this amount of mail? Sure, you can use spam filters and organize incoming mails in folders, as well as prioritize the mails. All these things are really helpful. Nonetheless, the number of high-prioritized mails is still much too high if you ask me. One of the worst things are those mails that ask you to decide for one of several options and send back your decision until a specific date. If you don't reply the sender will just assume that you have decided for one specific option. Tom DeMarco calls this kind of mail "Corporate SPAM". I consider corporate spam extremely offending.

What does that mean for our life as software engineers? How can we survive our daily professional work without being flooded by information overkill? Getting completely rid of e-mail? Sounds nice to some people but is completely infeasible. The only way is to organize e-mail, filter them, prioritize them, and come up with netiquettes in project teams to use e-mail efficiently in order to reduce noise and make everyone feel comfortable and productive.

to be continued ...

Wednesday, March 29, 2006

Meeting Syndrome

Now for something completely different. In most of the projects where I am involved I tend to spend a significant time in meetings. While face to face meetings are extremely important for decision making and team building, most of them are a waste of time. Tom DeMarco gave a talk last autumn in Cologne where he pointed out exactly that issue. He was consulting the two largest printer companies in the US. One of them proved to only achieve half of the productivity of their competitor. Thus, Tom was wondering whether the engineers in both companies had different levels of productivity. It turned out that the skill and experience levels were almost the same. Guess, the actual reason for the productivity loss? The less productive company was exactly the one with the most meetings. That does not imply, as Tom illustrated, that meetings are useless. It shows, however, that less but more efficient meetings are a better strategy. Tom also mentioned that the most successful meetings are the ones where people are expected to find a solution or decision. Meeting is over when goal is achieved, not sooner! The less successful meetings are those where people are just sharing information with each other without any particular objective or sometimes even without any agenda or time schedule. What I really dislike are all those meetings for which I don't know upfront what will be the agenda topics or goals in the meeting. Note, that general items such as status reports, action item reporting or planning of the next meeting don't count in this context! Tom DeMarco expressed the opinion that many meetings are only intended to let people communicate with their superior. Thus, the meeting is not a team-like approach but more a one-to-many experience. According to Tom it is much more productive when the boss is spending five minutes with each of the team members than spending two hours with the whole team. Another point which makes me crazy about meetings is when these meetings are either unmoderated or when there is a moderator who does not care whether people abide to the time span they got for their agenda topic. First of all, it is my time that gets wasted when the meeting exceeds its original deadline. Second, in many of those meetings I am really afraid to die or at least get severly injured by information overload.
Why is this relevant for software engineers? We as software architects spend most of the time communicating. It is essential that most of this time is dedicated to personal communication than with meetings.

To sum up: Meetings are important, but in order to stay productive meeting culture should focus on quality and efficiency instead of quantity. Meetings by themselves have no particular value. It is the human interaction that counts.

Tuesday, March 28, 2006

Gödel and Computer Science

For today, I have only one question where I'd like to get some feedback from all you out there in the infinite void of the Web universe. As you might know, there was an austrian mathematician Kurt Gödel who lived in the last century. He proved that for simple mathematical logic calculi everything that is true also can be computed (i.e., automatically derived from some basic axioms). For higher level quantification theories (i.e., where logic statements can reflect on predicates) this does not hold anymore. Thus, there must be truths which can not be computed by any processor. In the software world processors can be CPUs, Model Generators, Interpreters, Inference Engines. I am wondering whether there are any practical implications of this constraint? Or is this theoretically interesting but without any practical consequence whatsoever? So, what's your guess?

Monday, March 27, 2006

Software Architect Part II

I got some feedback from people asking me whether there ever can be a software architect with all the qualities and experiences I have introduced in a previous posting. Nonetheless, my opinion still remains the same :-) In some of my last projects we came up with interviews that checked for exactly the experiences and expertise described. And there were several software architects that passed the test. The problem these days is that no world-wide accepted qualification for software architects is available. Nor is there any role description. When and how will you know whether you can call yourself a software architect? I've provided all these little clues to help you here. All those experiences and expertise are particularly important to be recognized as a good software architect.
I'd like to introduce another perspective to shed more light on my points. What exactly are the typical activities of a software architect? Note: no guarantee for completeness given here.

From my viewpoint being an architect implies a really big workload. Architects need to:
  • help clarifying the business case and support SWOT analysis if required
  • help estimating required costs, resources, persons
  • help providing a domain language and a DSL (optional)
  • clarify all use cases and requirements as well as other open issues
  • build an interface between developers, testers, and other stakeholders
  • estimate iterations and increments
  • assess at least the most critical technology options
  • make sure the right tooling and technology portfolio is used within the project
  • communicate with system architects to gurantee proper interaction between infrastructure and software architecture
  • commit Commonality/Variability analysis when a Product Line Engineering approach is taken
  • develop and document an architecture vision that already contains strategic decisions. Note: this also implies that architects must decide whenever there are different options! Software architecture is not an automated, deterministic process without any degrees of freedom
  • help defining architectural guidelines for central issues such as error management
  • empower other developers and architects with respect to all architecture aspects and architecture concepts
  • take responsibility for all decisions and support partitioning and distributing tasks
  • design all structures and behaviors so that they meet requirements
  • supervise refinement and implementation of the architecture
  • support quality assurance for all parts implemented by distributed teams and integrated into the product. Note: outsourcing and offshoring can be considered just special cases here. I am not considering any cultural, or legal issues in this context
  • prove the feasability of a whole software architecture, of its parts, or aspects being leveraged by prototyping
  • review software architectures and re-engineer them if necessary
  • take test results and feedback as input for refactoring activities
  • communicate with key developers to make sure the tactical design does not violate the strategic architecture
  • simplify and refactor for injecting properties such as readability, extensibility
  • implement (but not on critical paths)
  • make sure re-usable assets are integrated
  • meet with customers to obtain feedback also during the project

All these points may be uncritical for small projects that use proven technology. This might imply that architects don't need that kind of high profile. Take a one-person project as the most extreme example. As soon as large projects and high risks are involved it is inevitable to make sure, architects can really handle all the aforementioned issues and cope with them. Hope, you can now see from this activity list, how all these required experiences and expertise were derived in my former posting.

And I definitely appreciate additional feedback and comments

Sunday, March 26, 2006

Patterns and the Real Life

In the last courses I gave on software architecture Í was using real life examples for some of the more complex patterns. Attendees told me that this approach had helped them significantly. Especially, patterns for concurrent and distributed systems are difficult to understand when introducing them in a pure software engineering context. Using real life examples for introducing patterns is not new. Several years ago Linda Rising initiated some OOPSLA workshops where participants had to provide real life examples for the patterns in the GoF and POSA books. I remember an asian guy whose wife was researching biochemistry and who came up with the example of a self healing organism in order to explain the Reflection pattern (POSA 1). Have you ever tried to understand the Reflection pattern without using a real life example? This is comparable to understanding quantum physics. Thus, the real life example approach should never be underestimated. I considered those real life example just a sort of entertainment for a long time but used them to explain to my wife and other people what the hell we were talking about in our books. In recent years, however, I recognized these real life pattern examples as a real value-add for pattern descriptions. I guess, the human brain requires some analogies to real world examples when processing abstract information (by the way, that is exactly the reason why quantum physics is almost impossible to understand, even after reading the Feynman lectures on physics).
Let me give an example. The Half-Sync/Half-Async pattern in POSA 2 deals with systems where asynchronous events are expected to trigger synchronous event processing in distributed systems. To separate both a queue is introduced by the pattern. This is exactly what will happen in a Burger restaurant. New customers will arrive and place their orders to an employee at the counter. This employee won't wait until all food items are ready but will tell some workers in the background to prepare the order, while taking the next order. This way, parallelism and thus efficiency is increasing. If these orders are processed in parallel, then their completion time is non-deterministic. How will the personal know which order is associated with which customer? This is where the ACT pattern is the only viable solution. ACT stands for Asynchronous Completion Token. Its solution structure is very simple: whenever an asynchronous operation is started, the operation is associated with a unique token either created by the requester or by the request processor. When the operation completes, the result is returned back together with the token. This way, the request and result can be easily associated with each other. In a Burger restaurant you'll get a number. As soon as your order is ready, someone will shout your number and you'll be able to pick up your food.
Needless to say that the same strategy works for Starbucks or beer gardens in Munich.
Hope, you'll see that these examples are particularly helpful to understand the patterns. Thus, my suggestion is to add real life examples to every pattern description, at least to those that are hard to understand. I am wondering whether we could also apply this idea in the opposite direction, i.e. taking real life examples for finding new software patterns.

Thursday, March 23, 2006

Architecture Qualities

When can we consider a given software architecture to be of high quality? Is there a unique, formalized answer that allows to build some kind of tool in the future that would just analyze an architecture description and measure its quality. One possible approach is to judge software architecture quality in terms of the requirements it meets with respect to the original goals and constraints of the software system under construction. If, for example, security was one of the original goals then a software architecture without any security-related entities would definitely not be called a good software architecture. With other words, quality of a software architecture is always relative to the problem going to be solved. But what about a problem specification that introduces contradicting, missing or vaguely specified requirements? Have you ever heard about the Garbage-In-Garbage-Out rule. A software architecture can only be as good as the problem specification it is based upon. Thus, software architecture quality can be considered as the fulfillment of a consistent and complete problem specification. One conclusion that can be drawn from this is that using metrics to measure software architecture quality does not make any sense. In addition, the precondition of a complete and consistent problem specification does not necessarily mean that you'll have to have such a specification when your project starts. Rather it implies that you should apply an agile method where the problem specification can possibly be refined anytime and the architecture must then be refined accordingly. Is that all that's important with respect to software architecture quality? What about readability and maintainability? Even if your software architecture can currently balance all existing forces, it might not be capable of addressing future requirements. Take remoting middleware as an example. Although .NET Remoting or Java RMI are excellent solutions for building business applications, they are quite unusable for the construction of embedded, real-time systems. How would you change RMI to support that kind of systems. You propably would have to completely re-engineer RMI. So, what? We are still on the right track, if we consider developmental qualities such as extensibility and maintainability as required parts of the problem specification. Unfortunately, the problem domain itself can be so complex that it is almost impossible to come up with a complete and consistent specification.
But there is even more. From my viewpoint, there some meta-qualities exist that are often ignored in requirements engineering. For example, symmetry and orthogonality. If the same problem occurs in different parts of your system your architecture should reveal the same solution in all of these parts. If different solutions are used for the same problem, understanding the architecture will be significantly more complex as you have to deal with more solution concepts. If the solutions are patterns that basically means: don't use different patterns to solve the same recurring problem except if there are very good reasons. Should we now try to increase pattern intensity for all of software systems we are going to build? I remember a guy talking to Erich Gamma and me on a conference in Vienna a few years ago. He told us that in his software system he has applied ALL patterns of the GoF book. Suppose, what our reaction was. Pattern density is not necessarily a sign for good software architectures.
Two of the most difficult issues in software architecture design are still unresolved:
  • What is the right structuring of a system. What granularity should one use? How many layers of abstraction or indirection are good or bad?
  • How could we systematically inject operational and developmental qualities in a software architecture?

Unfortunately, there are myriads of ways to solve the same problem. And among them are still myriads of good solutions. One reason for this is that many cross-cutting concerns appear in software design, Thus, it turns out to be impossible to decide exactly and determistically what entities we should interweave, which ones to separate, and what kind of relationships to introduce between them.

Nonetheless, there are good and bad smells. For example: Experienced programmers will recognize whether a software architecture reveals specific properties such as symmetry. They will understand whether a given structuring is good or bad.

Maybe, this illustrates that software design represents a combination of different areas: engineering, science, and art. It reveals some mathematical aspects but in the end intuition and experience are equally important. As a consequence, outsouring software architecture design is always a bad idea.

In other words, being a good software architect is what you should strive for.

Tuesday, March 21, 2006

Why Patterns have failed

Publishers and conference managers are interested to sell huge numbers of books and conference tickets. Software patterns are still a perfect means for attracting people. Thus, no one is interested to shed some light on the truth. Hence, I guess, most of you did not even suspect that Software Patterns have turned out to be a complete failure. Don't let these people fool you.

Here I have listed some clues and reasons for you:
  • Patterns as Brian Foote ones said are an "aggressive disregard of originality". They are discovered and not invented. As a consequence, there is not much space for innovative research in this area. Want to cover patterns in your Master thesis or Ph.D? It becomes obvious that researchers should forget about patterns.
  • Your boss told you to document your domain expertise using patterns? Good luck! Why do you think you were told to document your knowledge explicitely? Is it because your boss wants to enjoy your impressing capabilites? No, some people from foreign countries are supposed to take your position very soon. When your pattern documentation is completed, you will lose your job.
  • Do you know that IPR issues are essential for today's businesses? Guess, where someone will look to get information about your innovations. The architecture document and the code are potential places! If one of these is easily readable, it will also be easy to steal your ideas. Better encrypt your architecture by using unsymmetric and complex design instead of easing the life of industry spies with patterns.
  • Whenever you are encountering a new domain, you will make the experience of cultural shock. Domain experts tend to use some cryptic domain languages. There are two reasons for this: (1) no expert is interested to be understood by the rest of the world and (2) When domain experts speak with each other they need abstraction in order not to be confronted with reality. Patterns are the cryptic entities of the software engineering domain.
  • You will definitely know that patterns are considered blue prints and not implementations. They give you infinite numbers of implementation options and variants. In other word, patterns hide a high degree of uncertainty behind a deterministic facade. Thus, implementing patterns might make your life a complete nightmare.
  • What do you think about Karl Marx and George Orwell? Equality is an important issue in each society but striving for total equality might be completely counterproductive. Using the same patterns everywhere is like striving for total equality. Draw your own conclusions!
  • Did you ever wonder why the authors of the seminal book on design patterns were coined "Gang of Four"? This term does not reveal positive associations from a historical perspective. Do you really think, they called it this way incidentally?

There are even more arguments available. Maybe, someone else could make some suggestions and come up with additional disclosures. I hope, you now can see the whole story. There is some confederacy going on in the background. Everything is backed by a secret society called the Hillside Group. The conspirers meet in what they call xxxPLoP conferences. Their ultimate goal is world domination. Be cautious and trust nobody!

Saturday, March 18, 2006

What the hell is Web 2.0?

To continue with the discussion of my last posting, I'd like to discuss Web 2.0. As most of you will know, this term has been around for several months now. If you ask people what that could be, they will mostly refer you to new technologies such as AJAX. Applications like Google Suggest are prominent known uses for AJAX (Asynchronous JavaScript And XML). The AJAX concept however is pretty old. There were many Web applications available that used the same kind of approach. Take Web-based mail applications as an example. The concept behind AJAX consists basically of hiding all communication activities between clients and servers by introducing a kind of business delegate. Well, that business delegate is implemented using JavaScript when you use a HTML client aka browser.
Originally, Web 2.0 was just a name for a series on Web conferences. Wikipedia (see http://en.wikipedia.org/wiki/Web_2.0) defines Web 2.0 as follows

[...] Web 2.0 applications often use a combination of techniques devised in the late 1990s, including public web service APIs (dating from 1998), Ajax (1998), and web syndication (1997). They often allow for mass publishing (web-based social software). The term may include blogs and wikis [...]

If you read this definition, then it becomes obvious that this buzzword is another example for an attempt to keep the IT technology market continuously running.

On the positive side, the Web 2.0 hype makes people from different companies think on old but proven technologies so that they can be leveraged in Web applications more efficiently and effectively. As another side effect this might lead to standards. Standards despite of their liabilities such as being slow or being least common denominators at least foster developer productivity because we don't have to deal with dozens of incompatible and proprietary solutions.

Hence, Web 2.0 may boost our set of possibilities and capabilities. Take it, forget the hype, and use it to your own advantage.

At the end, that is what really counts

Monday, March 13, 2006

The Open Hype Scale

To earn money in the [IT] industry one of the potential preconditions is the availability of innovative solutions in your products. Innovative can mean a lot of things, for example faster, better, easier, or nicer. Only seldomly this kind of innovation improves or extends the functionality. Application domains have turned out to be very stable over time. So, what could a text processor do more than editing, spelling checking, or printing? On the other side, it is impossible for a vendor to publish a product and then live with it for years, without any regular updates. Same for other industries. What if your car would be the nicest, fastest, energy-saving car in the world with all features anyone could ever wish and what if this car could never be damaged by accident or age? Consider the truth: No one would ever buy a new car anymore. End of car industry. Point. Same holds for software development. That's why your new brand of software tools support MDA, UML, SOA, AJAX, VIIV, you name it. That doesn't necessarily mean that software vendors themselves invent all of these technologies, each of them promising a cure for all world problems. But if someone comes up with such a potential technology innovation, all vendors will jump on the wagon, no matter whether this technology is really useful. At the end of the day all of the software engineers, business accountants, and other personel must receive their cheques. So do I and so do you. It is essential for the IT industry to always improve and innovate, even if some of these improvements might be questionable. But don't blame our industry. Do you really thing that all of these features in your mobile phone, LCD-TV or car are necessary? Nonetheless, there is one implication for us architects and engineers. Make sure that your management never gets addicted to a new hype. I remember management guys to be totally convinced that the invention of components and their wiring with graphical IDEs would solve everything. That were the stone ages, you might think. But just consider the current SOA Hype. Most hypes come in waves and even if they don't succeed they reappear again wearing other clothes. To be able to prevent the worst, look at all of these new technologies very soon and try to keep yourself informed about all their strengts and weaknesses. And we as engineers should also be careful not to enter the trap. It is easy to come addicted to all this stuff as it promises to improve our lives as architects and engineers. Be realistic!

Sunday, March 12, 2006

Software Architect's Role

I am back. It took quite some time until my last posting. This was due to the work load I had in the last twelve months. From now on I am trying to be more verbose.

One of the topics I encountered in the last months was when I gave a talk at the OOP 2006 conference in Munich. An attendee told me offline he was going to manage a big software project and wanted some recommendations how to hire the right people for the software architect's role in his team. So what is the qualification an architect needs for software engineering? Is it just mastering UML and MS Project? Is it mastership of patterns? Of course, all these experiences are important but there is more. Personally, I would identify the following areas where a software architect should have expertise:

Software Engineering Skills: It is really obvious, that a software architect should have deep knowledge about all relevant technologies in software engineering such as UML, patterns, frameworks, components, services, requirement analysis, OOP, etc. She or he must also know upcoming trends, for example Model-Driven and Aspect-Oriented Software, Domain Modeling, Feature Modeling.

Software process: In addition, it is also important to know how software projects are organized in terms of roles, disciplines, phases, and activities. Rational Unified Process, Agile Methods are absolutely important here. Think about Unit Testing, Refactoring, Iteration Planning. Of course, quality-assessment and improvement are essential, i.e. techniques such as as metrics or methods such as CMMi.

Project Management: An architect must make sure that a software project is well organized in terms of resources and participants. The reason for that is simple: A software project should also be considered a (partial) failure when it does not meet its deadlines or resource constraints. As architects are the people that have all the experiences and knowledge already illustrated, they should definitely take care about project management, even if there is a dedicated project manager. And keep in mind: organizational issues such as project members coming from different departments might also be disturbing and counterproductive for any project. There is an old saying "show me your architecture and I know your company organization". Thus, architects should be proactive.

Business Skills: Every software projects - well at least most of them - follow some business case, meaning that management expects some benefits from the software system being developed. Thus, it is important to calculate whether a software development project is able to achieve all of the financial objectives. If, for example, stringent requirements are expected to be met by a customer such as "build this complex embedded system with a group of unexperienced developers in three weeks" then an architect must be able to evaluate the propability that the project might succeed. And, of course, if soime important requirements are not met by a software architecture, this might imply significant losses. Take commercial systems (software or hardware) as an example that revealed inappropriate architectures. Such problems are not very well received by customers.

Technology Expertise: Architects are not those people who develop Powerpoint and Word documents full of bubbles that can't crash. Software engineering is not a discipline where you can develop high level visions from 10000 feet without ever checking whether the real-world technology choices are able to implement the visions. For example, a software architect who is completely unfamiliar with Java EE won't ever be able to come up with an architecture that meets all requirements. If the software architects are domain experts they might be able to define the appropriate decomposition and relationships. But ... they are unable to meet most of the operational and developmental requirements. Believe me, most failed projects I've seen so far often failed exactly for that very reason. So, always keep up to date with the newest middleware, platforms, programming languages, paradigms.

Implementation Expertise: Architect always implements. That basically means, "eat your own dog food". An architect must design an implementable software architecture. And she or he must be able to refactor when necessary. Thus, an architect needs to participate in implementing and unit-testing. It is definitely not recommendable to introduce a BDUF (Big Design Upfront) approach where a couple of software architects sets up some architectural visions behind closed doors in a few weeks and then throws the architecture specification over the fence to the developers without being involved in subsequent development phases. If you don't trust your architecture you shouldn't be a software architect, anyway.

Social skills, the most important being communication. As an architect you will talk to customers, team members, business accountants, and all the other stakeholders. It requires huge efforts to keep all of them motivated and synchronized. For this purpose, an architect's daily agenda consists mainly of talking to other people. An architect must clarify requirements with customers, set the scope of the domain, design architectures with other architects, supervise the development team so that there is no deviation between architecture and implementation, make sure the testers are tightly integrated. And even more. Architects should motivate others and convince them.

As a consequence, someone who just has received an university degree, no matter how smart and knowledgeable, will not be able to be a software architect in the beginning. This requires years of experience.

There are even more issues here. But as always, space is void and can never be filled completely. Any further suggestions what makes up a good software architect?