SEACON 2011 Architecture Day in Hamburg

This year the German event SEACON 2011 is taking place from 27th to 29th June. The venue is the Atlantik-Hotel in Hamburg, a very nice location famous for its permanent guest, German Pop/Rock Tycoon Udo Lindenberg.

I was responsible to organize the architecture day on 29th June. And we managed to get excellent speakers covering a lot of interesting  architecture topics. Jim Webber from Thoughtworks is going to give the keynote address: “Lessons learned in large HTTP-centric Systems”. Attendees will also have the opportunity to visit half-day tutorials on Dynamic Software Analysis (Johannes Siedersleben) and Software Quality (Erik Dörnenburg). In addition, the agenda is filled with excellent talks by renowned speakers and practitioners.

For software architects this will be a highlight in 2011. The event SEACON Architecture Day might become an established event on Software Architecture. It is up to you. Attend and enjoy!

The hard way to Distribution and Concurrency

The Web is hot. So is Cloud Computing. And Multicores might even become hotter. Whenever there are additional capabilities, software developers will immediately start to exploit them. Why else do we always require new hardware for new operating system versions?

In particular, dealing with distribution and concurrency is pretty simple. Just start a thread in Java or .NET, and the force will be instantaneously with you. Or click on some menu items of your favorite tools to distribute functionality elegantly across the network. 

Unfortunately, many projects fail in providing distributed or concurrent functionality. How comes all these nice programming platforms and libraries don’t suffice?

And the answer of course is software architecture. Especially for cross-cutting concerns such as distribution and concurrency the programming level offers the wrong abstraction layer. It is not sufficient to address concurrency programmatically. Foremost, we must address the architectural layer, then the design aspects before thinking about programming idioms.

Let me investigate concurrency as an example. For this purpose I’ll consider different domains with respect to concurrency:

• In home automation systems there is a large variability of different services with various durations. Think about switching lights on/off, or controlling the climate system. Typical home automation systems are reactive systems that use a central component for event notification and event handling. As many events might occur in parallel,  the Concurrent Reactor Pattern offers the right architectural approach.
• For an image processing system we need a completely different architecture. Here, image data is created, processed and transmitted between subsequent components.  Hence, the Pipes & Filters architecture pattern with concurrent filters fits nicely into the domain.
• When dealing with control systems or Web Servers with asynchronous events arriving all the time that must be handled in a synchronous way, a Half Sync / Half Async architecture provides a message queue which decouples the synchronous from the asynchronous world.
• In Servers that process single messages such as Log Servers, a Leader/Followers patterns applies perfectly. The leader is in charge of receiving the next event. If it receives an event, it promotes one thread in the followers chain to the next leader, while becoming itself a worker that after work completion will become a follower. A nice example for a self-organizing infrastructure.
• In a warehouse system we’d like various actors to store or load items at the same time. Thus, the warehouse core cannot only provide one single interface. Instead, multiple interfaces operate on the warehouse core logic. They can synchronize their data by using a Half Object plus Protocol pattern. 
• In a mobile network or an AI system multiple, concurrent agents may need to coordinate their activities to achieve a common goal. For such systems, patterns like the Blackboard pattern are applicable.
• For divide-and-conquer processing the Master/Slave pattern helps structuring our Map/Reduce problem to sets of interacting components, some of them responsible for the Map, most of them responsible for the Reduce functionality.

We got various example domains, all of which had to use a different concurrency architecture approach due to their diversity of forces such as existence of sessions, variety of services, synchronization necessities, and many others.

Our first conclusion is: different problem contexts require different concurrency architectures. There can’t be a single concurrency architecture for all problems. In addition, we need to deal with architectural issues before introducing finer grained design aspects.

For the next step of refinement it is essential to solve issues at the design level. Think about Half Sync / Half Async where we need to determine  how to organize the synchronous worker threads of the Half Sync layer and how to let them access the shared queue of incoming events in a coordinated way, to address only a few aspects.

Therefore, we need to ask questions like:

• How will we manage shared resources such as threads or connections? Thread pools and other design tactics come into my mind.
• How do we efficiently deal with sharing and locking of data? We might use monitor objects or Software Transactional Memory approaches.
• How can concurrent entities communicate with each other and coordinate each other? Actor-based programming  patterns are one option here.
• How can we efficiently access data? Think about caching, eager evaluation and lazy evaluation?
• Is there a way to efficiently handle I/O? Asynchronous processing such as Proactors, Active Objects, or Asynchronous Completion Tokens come to our rescue.

Basically, all of those design aspects are on the component level. Fortunately, there are many design patterns that help us in implementing this abstraction level.

Our second conclusion: whenever we conduct fine grained design (i.e, tactical design), design patterns and design tactics offer guidance. 

Only in the final step, we are going to leverage all programming language and library features. Eventually, we end up using methods for thread creation and management, mutexes, agents, semaphores, and all those other idiomatic features modern programming platforms provide.

In the best case, a DSL and/or a library could help increasing the abstraction layer of the implementation by hiding details of the underlying hardware system. Think about OpenMP, PLINQ, Akka STM, and many other solutions.

What we learn from this example is that cross-cutting concerns like infrastructure or quality attributes require a sound fundamental architectural approach which is refined using design tactics and design patterns, and then implemented with the available tools offered by the programming platform. For instance, creating a concurrent solution requires an architecture-first instead of an implementation-first approach. For the same reason, SOA approaches favor interface-first over implementation-first styles.

Sure, programming might be much more enjoyable and exciting than drawing architectural boxes and lines, but at the end ad-hoc implementation of cross-cutting concerns causes more harm than fun. And don’t forget: creating a sound software architecture is sexy – at least for software engineers.

Where we can meet

I will participate in the OOP Conference 2011 in Munich which starts on Monday, the 24th January.

• Tuesday morning I will talk on Design Tactics
• Tuesday evening I will present Actor-based Programming
• Wednesday evening my Scala tutorial will take place
• Tuesday and Thursday I will also have the Meet-the-Editor events each 30 minutes where you might talk to me in my role as editor-in-chief of JavaSPEKTRUM
• Tuesday evening I will act in the IT Stammtisch organized by the unique and nice Nikolai Josuttis. So, if you would like to see me very uncomfortable, you should definitely attend this panel show.

I will also attend the QCon 2011 in London (7th to 11th March) where I will

• have a tutorial on functional programming with Scala and F# on monday
• host a track on Software Architecture Improvement at wednesday (talking myself in the introductory session)
• present the Onion model in Floyd’s track about models

Hope to meet you!

The Beauty and Quality of Software

In my spare time I often enjoy digital photography. Capturing people, streets, buildings, wildlife or landscapes involves a lot of fun and is a relaxing experience. After taking photos and when processing images with Adobe Lightroom, I am used to differentiate those pictures I consider decent from those I am going to throw away. The obvious question for a photographer is whether there are some guidelines or indicators that help taking better photos and also help rating photos. And, indeed, there actually are such quality indicators.

Independent of the content of a picture, there exist some common properties I consider important when rating a photo e.g.,

• Does the photo clearly focus on one specific object or does it rather confuse by showing various things without any focus?
• Is the horizon depicted horizontally? Even if there is only 1 degree deviation the human watcher won’t feel comfortable.
• Are there some interesting kinds of symmetries in the image?
• Does the photo contain a chaos of many different things or does it concentrate on one specific detail?

There are many other indicators for the internal quality such as the rule of thirds. Interestingly these rules are really helpful. However, they are not proofs for good or bad quality of a photo, but mere indicators. For example, sometimes deliberately breaking symmetry or violating the rule of thirds may lead to much more expressive photos. No rule without exception.

How does all of that relate to software architecture? The question is whether we can apply some indicators to get a first impression about the internal quality of an architecture, something I consider as architectural beauty. When I once read Dave Cutler’s book on the design of Windows NT, it really opened my eyes. All parts of the architectural design were easy to understand. The architecture was expressed in a kind of idiomatic way, where the same principles have been applied to different parts. The partitioning of the overall operating system in different layers and components with clear responsibilities helped me grasping the details easily. On the other hand, I have also experienced bad architectures with over-generic designs, where it took me weeks to get the slightest clue what they were trying to implement. So are there quality indicators for good or bad design?

Obviously, there must be such indicators. Otherwise, the widespread addiction to metrics wouldn’t make too much sense. Metrics combined with CQM tools (CQM = Code Quality Management) and Architecture Analysis tools can reveal issues such as insufficient cohesion or coupling. Metrics however must always be considered relative! For instance, there is no rules of thumb whether 5k LOCs are good or bad. It might be good in one case, but bad in another. Applying the McCabe Cyclomatic Complexity can lead to wrong conclusions (If you don’t believe me, calculate the cyclomatic complexity of an Observer scenario with 80 observers). Why is that? Metrics operate on syntactic level. Thus, they can only measure syntactic properties. They are not capable of dealing with semantics. Hence, all metrics must be set into the right context by human engineers. While a coupling of 5 itself might not be too valuable, the increase of the coupling from 5 to 10 after one iteration reveals a potential problem.

One of the more absolute indicators are architecture smells which also help figuring out the necessity of architecture refactoring. Let me give you a few examples:

• dependency cycles between architectural components imply that you cannot understand, test, change one component without addressing the other component in the cycle.
• Inexpressive component names prevent engineers to understand the architecture without digging deeper into more details.
• component responsibility overload means that a component is implementing too many different responsibilities preventing clear separation of concerns.
• Unnecessary indirection layers do not only negatively affect developmental qualities such as maintainability or extensibility but also operational quality attributes such as performance.
• Implicit dependencies often lead to a shift between desired architecture and implemented architecture. One notable example is violating strict layering, thus introducing unnecessary und unknown dependencies.
• Over-generic design:  When there are dozens of Strategy patterns (or proxies, visitors, etc.) in your design, this often denotes a clue for a potential problem lurking in your design. A Strategy pattern basically means “I have no clue but want to defer the decision to another place”. Applying many Strategy patterns mean, the architects had absolutely no clue. Instead of opening the system according to the Open/Close principle for some variability points, they applied Strategy unsystematically. This is the best way to achieve less flexibility and less performance.

Architecture smells can often be detected using architecture analysis (tools). If we think about these smells, we recognize that there must be something more that gives us indication about the internal quality/beauty of an architecture. Something which resembles which I addressed in the introduction on digital photography.

I can not speak for the whole community of software architects in general. However, here are the quality indicators that help me assessing a software architecture:

• Simplicity: A software architecture is simple when it addresses the requirements with the least possible number of design artifacts  (KiSS) and does not introduce accidental complexity by additional entities (aka design pearls) that are not required. There is the old rule: “an architecture is simple when you cannot remove anything without failing to meet some requirements”. Hence, a simple architecture is not a simplistic architecture. To achieve simplicity, one guideline is to root all architecture decisions in the architecturally relevant requirements. You can measure simplicity of an architecture using a simple test: ask the architect to introduce the architecture within at most 30 minutes only using a flip chart to draw the architectural baseline. I know, this also depends on the presentation skills of the architect but that’s another story. Nonetheless, a listener should be able to repeat the core architectural decisions. There is one caveat: Some “smart” software developers try getting rid of complexity by just hiding it in the guts of the implementation. For example, a system could theoretically consist of one single object with a doIt method. This is like some of the ESB and EAI products that just hide the mess behind some wrappers. This, however, does not introduce simplicity, but rather moves complexity to lower layers. The same simplicity test would definitely reveal that kind of bad trick.
• Expressiveness: An architecture is expressive when you can easily grasp the purpose of all entities and the whole architecture by looking at the architecture baseline. I don’t claim an architecture needs to be self-explanatory such as understanding it by just looking at the UML diagrams. This can’t work because diagrams fail to reveal leading design principles and the rationale of design decisions.  What I mean: Just looking at the architecture design should give you enough understanding to grasp the whole architecture vision by diving into some additional details. How can you achieve expressiveness? First of all, give expressive names to all entities, so that another engineer gets the idea. For all components make sure:  It should do one thing but it should do it right. And each component should only have one responsibility. Assigning lots of responsibilities to the same component leads to inexpressive architecture as do all responsibilities crosscutting through your design. Role-based design is a perfect tool to assign role-specific interfaces to components. Speaking about interfaces: Also add a contract to each interface that specified what (kind of protocol) the interface expects and what it provides. Aspect-oriented development may help dealing with crosscutting concerns. Another important issue in this context is to make all dependencies explicit in the architecture. That’s a no-brainer because implicit dependencies are simply not visible in the design so that an architecture reviewer won’t recognize them easily. Expressiveness can be verified by a simple telephone test. Ask an architect to explain his/her architecture via a 10 min telephone call. After the call you should be able to understand at least the coarse grained architecture.
• Behavioral Symmetry: Suppose, you got are viewing sequence  diagrams of a transaction-based enterprise system. Although there is a method invocation called beginTransaction, you’ll recognize no endTransaction, commit or rollback. Won’t you feel uncomfortable with such an observation? Symmetry of behavior is a good indication, something might be wrong with the design. As it holds for all quality indicators, it is not a proof for bad quality. For example, in a distributed system the remote server might allocate memory for a new result object, while the client is supposed to free it after use.
• Orthogonality or Structural Symmetry: I once had to review a software architecture where they used several solutions for the same problem. Hence, I required to understand all these various solutions to understand the architecture. Structural Symmetry means, you are leveraging the same solution for all instances of the same problem. Even if MFC provided a handful of string classes, you should not use all of them in the same application. This is particularly essential when dealing with crosscutting concerns such as error handling, tracing, logging. Imagine, each developer would introduce his/her own error management strategy. In this case you need to cope with broken structural symmetry. To avoid structural asymmetry, you may introduce guidelines and conventions  for crosscutting concerns or recurring problems. It is not sufficient to provide documents, though, but you should also enforce these guidelines actively in the project. When reviewing software architecture, ask for such guidelines and ask how they have been enforced.
• Pattern Density:  We all have heard about the “not invented here” syndrome and may even have experienced it ourselves. Patterns capture the knowledge of experts how to solve recurring problems. Thus, a wise engineer would rather apply a pattern instead of coming up with a homegrown solution. The more patterns have been used, the better. Wait a second! That is not quite true. In this context, by using  patterns I mean using them whenever they solve a problem you really have. Not like in the “hammer-and-nail” syndrome with an attitude like the following one: even I got no need for Observer in my system, I will change the system so that I can apply the only pattern I know. Applying patterns is not trivial. Patterns are not Lego building blocks you can add to your architecture. Rather, you need to adapt your design, map the pattern roles to your architecture and integrate all the patterns properly. In some hotspot areas of an architecture design, the same components may contribute to several patterns. Thus, even high pattern density could be malicious if the patterns have  not been integrated properly.
• Emergence: In an ant colony all individual elements reveal very “simple” behavior, but the whole colony seems to act smartly. Another example for the whole being more than the sum of its parts (Aristoteles). The Internet and the Web  are also good examples for this kind of emergence. They consist of simple elements such as Ethernet, TCP/IP, DNS, HTTP, SMTP, RSS which serve as the building blocks for a complex ecosystem. Emergence also means decentralization and self-organization. Many architects are completely addicted to centralization. Often when quality attributes such as scalability, availability or fault-tolerance have high priority such as in Cloud or P2P Computing, decentralized approaches are much more effective. Think of the Leader/Followers patterns as an example which introduces a self-organizing pool of threads that reduces resource competition by only allowing the leader to access the joint event source.  Another indirect consequence of emergence is that you shouldn’t strive for fat APIs that implement all methods, but just simple APIs providing methods with which you can  easily implement more advanced functionality.
• Partitioning and Spacing: The responsibilities in your design should be mapped systematically to components and subsystems. For example, subsystems should only contain functionality that reveals some semantic coherence. That is, the closer two different kinds of functionality are semantically related, the closer should they be aggregated in the design. Layering is a good way for separating different levels of abstractions with top layers being more abstract (close to the problem domain) and low layers being less abstract (close to the solution domain). Basically, you need some kind of systematic top-down design and problem decomposition so that the resulting subsystems and components will offer an adequate partitioning of responsibility and a sufficient spacing (different kinds of concerns should be mapped to different subsystems, layers, components). In addition, architecture entities should offer a clean partitioning into role-based interfaces instead of relying on one bloated interface per component or subsystem. An example could be an additional interface that enhances the testability of the system.  Or an interface for configuring a component. If possible commonly used interfaces such as configuration interfaces, should be defined uniformly for the whole system. Otherwise, configuration would need to deal with many different configuration approaches (see the Component Configurator Pattern for more details).

There are definitely  some more qualities I could add to this list of indicators. However, in my experience these aforementioned quality indicators already offer you an excellent mental tool for assessing the internal quality and thus the beauty of a software architecture.

Mind the gap:  As mentioned several times, each of these quality indicators just serves as an indicator, not as a proof. But if the design under consideration does not provide one of these internal quality properties, the responsible architects should at least offer you a good and convincing rationale why this property is missing.

Needless to say, that quality indicators should not only be leveraged for assessing existing software designs. Likewise, you should consider them valuable when designing a new software architecture.

Qcon 2011 in London: 100 British Pounds discount

I will host a track on Architecture Improvements at QCon 2011 with a row of excellent speakers. In addition, I will give a talk on Software Architecture Design as well as a tutorial on Functional Programming with Scala and F#.

If you’re interested in attending the conference just use the promo code STAL100 for receiving a 100 BP discount.

 

Here’s the URL for the QCon.

Small is better - really?

In different panels and discussions I often hear programming languages such as Scala are not a good idea due to their "language size". Less syntax is better to learn a language which is why, for example, Clojure is superior to Scala. Often, the proponents of this argument point to languages such as C++, C# or Java as negative examples. Resembles software architecture a lot, doesn't it? Honestly, I object to this conclusion. Take languages such as brainf*ckr or the Turing Machine. Very simple syntax indeed, but would you consider them powerful languages? Definitely not! In my viewpoint it is not the size that matters but the noise-to-signal-ratio. If you need to express a solution for a given problem, how much efforts do you need to find and articulate the solution using the language idioms. The more appropriate and concise idioms are available that support your solution, the better. As in software architecture some qualities are important when judging languages:

• appropriateness to solve a specific class of problems
• simplicity
• expressiveness
• orthogonality of language structures
• approach of least surprise
• emergence of features
• avoiding implcit dependencies and influences (such as temporaries in C++)
• availability of powerful language idioms

By the way, these qualities are also applicable to software architectures. This is no surprise because we are speaking about architecture of programming languages here.

There are several conclusions we could draw from that qualities:

• Even if a language offers most of these qualities at the beginning, it might erode over time when it is evolved in an unsystematic or improper way
• Several problem classes might imply various languages. So, we have to choice between a best of breed approach for each problem class as a polyglot programmer would prefer, or we could try to indentify some multi-paradigm language. Note, however, that multi-paradigm languages are those which are often very likely to erode, especially when more and more paradigms are added as an afterthought. However, this is a risk, not a predefined fact. For instance, multiple paradigms were cleanly integrated in Scala from day one, while C++ started as a structured language with  additional classes.
• The language core might be excellent but it won't be helpful if the support libraries and tools do not reveal the same qualities mentioned above. A fool with a tool is still a fool. So, even for the best languages, APIs of libraries need to integrate tightly.
• If you follow the Pragmatic Programmers advice to learn a new language every year, you will also be able to extend your knowledge about new solution paradigms and idioms. This will drastically improve your skills as programmer and architect. Mind the gap: it is not sufficient to just learn a language, you also need to practice it for a while.

Unfortunately, using multiple languages at the same time might be a bad idea. I experienced a project where people invented their own language. After a while, only one expert was left who did understand the language which was not that comfortable for the organization. If you plan to use multiple languages, think about the problems at hand. If they can be better solved using multiple languages, go for it, but make sure, several people in your team know these languages. If possible, use languages that run on either the DLR/CLR or the JVM, because then they will share the same SDKs and (often) even the same tools. Make yourself aware that you ARE already using several languages such as HTML5, XML, SQL, DSLs and so forth. We already have become polyglot programmers. But that doesn't mean you should strive for large numbers of languages in your projects.

Personally, I have learned a lot of languages in my career: x86-ASM, Pascal, Modula2, VB, Java, C, C++, Ruby, Lisp, Clojure, Scala, C#, F#, CIP, Haskell, D, Axum, ... All of these languages have their purpose and their strengths, but also their weaknesses and problem areas they cannot address well. Some of them are large and easy to learn such as Scala, while others are small and difficult to learn such as Lisp which is why they invented Scheme :-)

Size does not necessarily matter, the problem you are going to solve does.

Books, books, books and tools

In vacations I often dedicate some time to reading new interesting books as well as experimenting with programming languages. For a software architect I consider it essential to stay in sync with technology evolution. Honestly, I do not believe in ivory tower architects who just know UML and Powerpoint. So, what are my plans for this christmas season?

• One of my favorites is Martin Fowler’s and Rebecca Parson’s book “Domain-Specific Languages”! This book explains DSLs but also educates how to practically leverage internal and external DSLs.
• From the Pragmatic Programmers I received “Seven Languages in Seven Weeks” by Bruce A. who provides a hands-on tour on Clojure, Haskell, Io, Prolog, Scala, Erlang, and Ruby. If you are a fan of programming languages, this is definitely a good read.
• Of course, I am also enjoying fiction books as well such as Michael Crichton’s “The Great Train Robbery”. I recently started to read Crichton’s novels and really like his work. In addition, I am reading some layman books on the universe and quantum physics. I am currently  too lazy to reread Feynman’s Lecture Notes on Physics.
• For my preparation for OOP 2011 where I will give three talks, one on actor-based programming, one on Scala, and one on design tactics, I have programmed a lot with Scala 2.8.1, sbt, and Akka.   But I am also currently trying to dive more into Clojure. I used Lisp in the university, recovered my knowledge a few years ago, and now starting to get familiar with Clojure. This is where monads come in. Interestingly, there are many definitions and explanations of monads but none of the is really decent. Maybe, I will give a try in the future, and also fail

So many new things to learn in such little time. Unfortunately, the times are over where I could learn the whole JDK or .NET in a weekend. Software engineers have to get more and more focused. It is even impossible to keep yourself up-to-date with enterprise technologies alone, or concurrency techniques, or mobile computing, you name it. Nonetheless, systems are gaining more and more complexity, becoming bigger, using more technologies. Interestingly, we are always testing our limits. No matter how excellent our tools are, we try building systems that are more complex than our skills allow. With other words, technology and complexity are always ahead of us. Software architectures do not only fail to master inherent complexity but become beasts filled with accidental complexity. Our capabilities do not scale with our problems. Did I mention Henri Petroski’s book “Learning from Failure”. The essence of this book is that we always will go over the limits, but the best thing to survive is to learn from failure. This basically resembles what Thomas Kuhn once said about technology evolution and revolution.

Fortunately, it also means a lot of fun to learn new things or technologies, try them out, learn their strengths and weaknesses, increase your mental toolset. It opens the horizon to some new ideas and ways. This is why learning is one of the most important skills for software engineers.  

So, what are your plans for the rest of this year?

Understanding software architecture

While creating software architecture is a  very creative activity which I really enjoy, analyzing existing software systems is much harder and happening frequently.

There are several reasons for this observation:

• Often, the designed and the implemented architecture differ dramatically. In these cases, the documents are nothing than fairy tales.
• In some cases, I am encountering a kind of documentation hell. The information surely can be found somewhere, but it is almost impossible to figure out what could be the right document. I am here to find not to search.
• Another “highlight” are insufficient abstraction levels in the documentation. No one can read end even less persons can understand an architecture document that covers 1000 pages. Likewise, the other extreme isn’t helpful. For example, I once received a software architecture document consisting of one page filled with UML diagrams. 
• What I really dislike are documents that only show the WHAT but not the WHY. However, I definitely need to know the rationale of all those decisions. All design pears that pop up from the void of the universe are worthless to all those mediocre reviewers who must understand them.

So how you could approach such an endeavor.

• Firstly, get an idea about the domain, the development organization and the business, that is, the ecosystem, in which the software system lives.
• Get an overview of the coarse grained architecture, from the user view and the developer view. I prefer strategic architecture design documents, guidelines, programming conventions, presentation slides, and even marketing material for this purpose.
• Get a knowledge about the tools and technologies of the solution domains that have been used.
• Interview some designers and developers in order to understand the implemented architecture (and the designed architecture) as well as its weaknesses and strengths. The more documents additionally capture the design essence the better. Both information sources will provide you with a good idea about the architecture.
• Use architecture analysis tools to understand the internal quality of the systems (and also its pain points).
• Try to change or implement yourself in the codebase of the system to stay close to the ground and thus to reality.
• At the end, you should yourself be able to explain the architecture to other persons.

Basically, what you need to do to understand an existing software system is similar to conducting the first phases of an architecture review.

Don’t be shy and ask any questions that come to your mind. Often, these Q&A settings help architects figure out potential problems in their design, and help you understand the architecture. So, it can lead to a Win-Win situation.

Software Architecture Reviews

Software Architecture Reviews represent one of the most important approaches for introspecting software design. The best way for introducing such assessments is to establish regular evaluation workshops in all iterations. Architects and designers will review the software architecture focusing on internal qualities such as unavailability of dependency cycles and quality attributes such as performance or modifiability. If they find any issues, they will define action items and means for getting rid of the challenges (also known as problems). My own experience shows that this is definitely the recommended way, because regular reviews will detect any issues as early as possible before they cause further harm such as accidental complexity or design erosion. If introspection tools such as Odasa or SotoArc are available, detecting internal quality issues is often surprisingly easy. And if architects compare quality attributes with architecture decisions in early stages, they will reduce the probability that an important nonfunctional requirement was neglected or designed in a wrong way.

Unfortunately, many architecture reviews are initiated very late. This makes problem detection and architecture refactoring much more complex. Nonetheless, such architecture reviews are always a good solution to get rid of architectural problems, especially when the organization could not handle such issues successfully in the development project. These reviews should have a clear scope. Otherwise, the review will take place for large time frames which is not acceptable in most cases. Scoping in this context means to define a main objective of the review as well as 2-3 key areas where potential problems supposedly lurk in the guts of the design. The result of such a "large" review should be a document with key findings, e.g., weaknesses and strength of the architecture. However, it should not only contain the weaknesses but also appropriate solutions for addressing these weaknesses. Some review methods don't consider solutions mandatory in such a report. I definitely do. Even more, I consider ways to get rid of the weaknesses the most important result of a review.

Such a report should reveal the following structure:

• Scope of the review and review method
  
• Documents (project documents) and information used (for example: interviews, test plans, source code, demos)
  
• Executive Summary
  
• Overview of the software architecture under review
  
• Findings and Recommended Solutions
  
• Summary
  

While regular, iterative reviews can be conducted by project members such as architects and developers, larger review projects should be done by "external" reviewers. There are two main reasons for this recommendation: First of all, external reviewers often see more. And secondly, higher management representatives are often inclined to accept external recommendations more than the ones from their own project members.

Interestingly, an architecture review is not constrained to software architecture challenges. It might also reveal problems in the organization, development processes, roles and responsibilities, communication, technologies, tools, business. To be honest, only rarely the results will address design problems only. That is another reason why architecture reviews should have external reviewers.

There is a whole selection of different review methods such as CBAM, SAAM, ATAM documented in literature. We at Siemens have developed our own method called Experience-based Reviews. While the former 3 methods are scenario-based (which I will explain in a later blog post), experience based methods are driven by the experience of the reviewers and are less formalized.

Such a review normally consists of the following phases:

• A Kickoff Workshop where reviewers and project stakeholders meet to introduce the review method, illustrate the software architecture, and define the review scope
  
• In the Information Collection phase, all available information is collected by the reviewers such as documents, test plans, source code, demos. Further information is collected by conducting interviews with the project's stakeholders, 1 hour for each and constrained to one interviewee. Information is kept anonymous in order to establish a relationship of trust between reviewers and interviewees.
  
• In the Evaluation phase all information is evaluated. Strengths, weaknesses, opportunities, and risks are defined by the reviewers. So are all the possible solutions for the weaknesses and risks. At the end the reviewers create a review report draft structured as mentioned above. They then send this report to all stakeholders, leverage their feedback, and disseminate the final report.
  
• A Final Workshop helps to summarize the key findings and discuss open issues.
  

This approach works extremely well if the reviewers are experienced. We normally will use the Master/Apprentice pattern to educate architects how to conduct an experience-based review. Senior reviewers will lead the reviews and teach junior reviewers by training-on-the-job.

All of these approaches define qualitative assessment methods. In addition, architects and developers should also leverage quantitative methods when actually creating design and implementation. Tools for quantitative assessment consist of but are not limited to feasibility prototypes, simulations and metrics.

Architecture assessment methods are a bit like testing or refactoring. Many engineers think they can survive a project without these disciplines. But in all but trivial projects this proves to be a wrong assumption. The more you invest in early and regular testing, refactoring, architecture assessment efforts, the better your RoI will be. Do less in the beginning, pay more in the end!

 

I have been upgraded

Because I got so many questions on this: Most recently, my name got a little bit longer. The University of Groningen in the Netherlands (RuG, Rijksuniversiteit Groningen) has appointed me Professor Extraordinarius. Thus, I am now member of the Bernoulli Institute of Mathematics and Natural Sciences where I will give lectures and more.

If you now ask: this is in addition to my main profession at Siemens which by the way supported me in achieving the position. Of course, I am very glad to cooperate with Paris Avgeriou who is a professor himself at the RUG. Another personal friend and promoter, Jan Bosch, is member of the same institute. Needless to say, that Paris and Jan are internationally renowned software architecture experts with whom it is a lot of fun to work together. I am quite impressed by their knowledge and expertise, they also use to distribute to their students.

My appointment was greatly supported by Nicolai Petkov, another professor at RuG, whom I really like and respect. After the deans of all Computer Science Instititutes in other Dutch Universities  had agreed to my appointment after checking my C/V and my References I finally received the appointment letter.

The RuG is quite famous for its research on software engineering, in particular in Product Line Engineering. Not to forget all the RuG offspring such as Jan Ommering and a lot of other great Product Line gurus. I do not only enjoy that but also the Dutch mentality and great hospitality whenever I am in Groningen. Only a few weeks ago I’ve been there to give lectures to students as I already mentioned in my last blog post.

Thus, I will be able in the future to support Ph.D students, master students as well as postdocs. They may learn from my experiences, but I will also learn a lot from them. You could call that a Win-Win situation.

So that’s the big news.

Teaching Students in Software Architecture

This week I have been giving lectures on software architecture for master and PhD students at the Rijksuniversiteit in Groningen (RuG). It worked excellent and it was a lot of fun. I teached them systematic software architecture design, design tactics and introduced a pattern system for distributed and concurrent systems. What worked particularly well was the split into a presentation part and a tutorial part - each lasting 2 hours per day - where participants were designing example projects in groups of up to 5 people. This way, they would not only learn about architecture design, but also about group interaction. In fact, many problems in software architecture are caused by people issues such as lack of (efficient) communication.

They will also get a group assignment and an individual assignment for the time until end of June.The former one will comprise completing the architecture design and providing an architecture description while the individual assigment requires students to create a design essay on an architectural quality (in the context of their example project).

I received positive feedback that I did not introduce just a bunch of unrelated patterns but narrated a pattern story of how to apply patterns for the design of middleware and distributed systems. From my viewpoint, this is the best method to educate students that patterns are not just disconnected islands of code, at least they shouldn't be. Most of the power of patterns comes from using systems of patterns. Pattern languages would be even more attractive, but unfortunately there is only a bunch of them and the ones existing are really complex.

For the lecturer it is important to be constantly available for the exercise groups during the tutorial parts. I got a lot of questions about architecture design, modeling and documentation. It is pretty close to architecture enforcement. You cannot simply throw an (exercise) specification over the fence and expect attendees to exactly understand all issues. Management by walking around is much more effective.

As material for the course I relied on the excellent Software Architecture in Practice book, 2nd edition, by Len Bass,  Paul Clements and Rick Kazman and of course - needless to say - on our POSA book series (Pattern-oriented Software Architecture). Using POSA patterns is beneficial in that it shows  that the pattern community does not only depend on the GoF pattern book.

In summary, the mixture of conceptual parts and exercise parts works nicely to educate students about software architecture.
Looking forward to providing more lectures in ths future and keeping in touch with the students.

Functional Programming

In the last months I have intensified my skills in functional programming. My favourite programming languages have been Scala and F# so far. Only a few years ago I recovered my Lisp abilities when I had been ill for some weeks.  It is interesting how many functional languages currently evolve. Guess, it is closely related to the availabilty of virtual platforms such as the JVM and the CLR. No need to develop a whole bunch of APIs or IDEs anymore. just use the Java SDK or the .NET Framework Classes, and provide some plug-ins for Visual Studio, Eclipse, NetBeans, or any other IDE of your choice. A real paradise for language developers. Reminds me of the Eighties when all those OO languages appeared such as C++, Smalltalk, Objective C, or Eiffel. I anticipated several years ago that this would happen. Interesting, you might ask, but how is all this related to architecture? A lot if you ask me. Remember the book on object-oriented design patterns by Erich Gamma, Ralph Johnson, Richard Helm and the unforgettable John Vlissides. This book (as well as hopefully our POSA books) have changed the world in  that they brought software architecture in people’s minds. The same will happen through functional programming.

Functional programming enforces another way of thinking. Functions are first class entitites that can be assigned to values, passed as arguments to higher-order functions, or just created anonymously (called: closures). In pure functional programming there are no variables anymore, just immutable values. All of this was already specified by Church & all in the Thirties.  While the first functional languages such as Lisp, ML were difficult to understand for everyday programmers, new functional languages deviate from the academic approach and turn into hybrid languages that integrate object-oriented features, and permit mutability by supporting imperative programming styles.

Sometimes mutability is important such as for I/O and GUI operations or logging/tracing. But there are many situations where mutability is like shooting yourself in the foot. Think of race conditions in multithreaded applications. Most accidental complexity is caused by mutable state. No mutable state implies no race conditions. Or think of side-effects that make your application hard to understand and debug. These side-effects are simply missing in functional languages. Think of testing or validating your functionality. Guess why Joshua Bloch spends so much space in his book on Effective Java illustrating how to make Java classes immutable.

In contrast to object-oriented approaches functions are used to abstract functionality and function composition is the basic means of composing artifacts. That’s a perfect extension to object oriented abstraction and composition.  Hybrid languages such as Scala use this combination to make building internal DSLs surprisingly easy. For instance, the Actor functionality looks like an integral constituent of the language, while in fact it is just a library. If you wonder what actors are: they can be considered abstract tasks or objects running on their own threads. They do not provide any possibility for reading or changing their internal state. The only way to communicate with actors is by exchanging messages´with them. If these messages are immutable values, there is no chance of side-effects.  What a perfect way for implementing multithreaded applications.

Another famous example for functional features used to provide an internal DSL is LINQ (Language Integrated Query) in Microsoft .NET. Using extension functions, closures, anonymous types, and type inference helped to seamlessly integrate LINQ into languages such as C#.

Sometimes you need to deal with state. Languages like Clojure also support Shared Transactional Memory, a good way to encapsulate all state changes within transactions.

What makes the functional paradigma so effective, is it’s succinctness. As an example we’ll use F#:

let rec fib x =

     if x < 2 then 1

     else fib (x–1) + fib (x-2)

let results = Array.map fib [| 1 .. 40 |]

The code above introduces a recursive function for calculating fibonacci numbers, then builds an array containing the first fourty fibonacci numbers. Try this with Java, C# or C++.

Many problems can be described in a much more natural way using functional features.

Thus we already know several aspects where functional languages are beneficial for the architect:

• introducing internal DSLs
• Parallel and Asynchronous Programming (e.g, targeting Multicores)
• Implementing some parts of an architecture in a much more succinct way
• Improving testability
• Understandability of design (expressiveness, simplicity)

As mentioned previously, functional approaches do not represent a panacea. Neither does Object-Oriented Programming. But they  surely offer an excellent extension to imperative or object-oriented design and implementation. Consider them a perfect extension of Object-Orientation.

It is time to follow the advice of the Pragmatic Programmers and learn at least one additional language per year. My recommendation: focus on functional languages for this purpose.   

Implicit versus explicit

In a very larg telecommunications application I was supposed to review, the software architects had introduced a strict layering to shield different abstraction of functionality from each other. One of the lower layers comprised the database system, while the topmost layer consisted of the application UIs. Whenever something in the DBMS layer changed, the UIs broke. This shouldn’t happen according to the software architecture document. Thus, I asked the UI developer whether he abided by the strict layering. His answer was no, because he got the order to optimize performance which required him to access the data directly. What we see, is a typical example of implicit design decisions. When designers or developers break the architecture or use it in an unanticipated way without communicating the issue to software architects, this will inevitably lead to severe problems which are typically hard or even impossible to detect.

In another project the development organization built a GUI for a monitoring framework. After the application had been relaesed, customers started to complain about a feature they were expecting but which was missing. This illustrates what happens if requirements engineering or customers have implicit expectations. Unfortunately, these requirements change over the years. While 20 years ago almost every developer was satisfied with command line compilers and had to explicitly specify she would prefer more IDE-like environments, today this feature is considered an implicit requirement. You don’t need to mention it explicitely anymore. KANO-Analysis is a perfect tool for such investigations. For programming environments this is very obvious bút what if you’re working in a domain where you don’t know about all these hidden requirements. Thus, mind all invisiple traps.

There are two conclusions we can draw from these examples:

1. In a software development project, any information should be made explicit, even facts we normally believe every stakeholder should know. If communication among stakeholders is the most important asset in such projects, nothing shall remain implicit. Implicit information is unavailable information!
2. In every project we need enforcement. Requirements engineering must enforce that the development team really knows and uses all requirements. In a specification all implicit requirements must be turned into explicit requirements, because implicit requirements are typically swallowed by black holes. Software architects must enforce the architecture by closely cooperating with developers and must ensure quality by closely cooperating with testers.

Work based on implicit information can only possibly work in small teams that use an agile approach and where every team member has an explicit knowledge of implicit information. But do you dare to take the risk that all information gaps are really closed?

Always be an explicit software architect without a hidden agenda!

Over the Fence

It is surprising how many software architects still create their design, throw it over the fence, and then expect others to follow the divine strategies implicitely hidden behind the lines and diagrams.

This behavior also holds for different teams with key developers or subsystem architects that are ought to coordinate design activities, while in practice they are happily inhabiting their isolated islands.

I’ll never forget a telecommunications project where the architects had introduced excellent guidelines and a decent strategic design. However, I was called for an architecture review, because the implementation had turned out to be not quite decent, to say the least. When I searched for the root of all problems, it became clear very soon that no one in the project had really cared about any of the architecture guidelines. And even worse, the strictly layered architecture design had been seriously violated. For example, whenever the database schemas were evolved, the clients broke, although they were totally isolated from each other by a “firewall” of 3 layers. The key developer of the GUI told me in an interview, that he considered performance optimization as his primary directive so that it semed to be totally acceptable to directly access the database.

We can learn from these war stories that good communication is the (most) critical aspect in any project. Without good architecture enforcement, there is no good architecture. But how can architects  pragmatically ensure high quality as well as conformance with the architecture design? Management by walking around does the trick. You should frequently visit the different teams and make sure, they really understand the infinite wisdom and essence in your architecture design and architecture guidelines. This is the best way to get their buy-in and support. In addition, you ‘ll detect any architectural shift early. In particular, when developers add accidental complexity by introducing design pearls that are not motivated by requirements. I wholeheartedly believe, agility implicitely requires architects to work this way. But it is not the case that architects are inerrable, (although we’d really like to see the universe from this perspective). Sometimes, even clever wizards fail. As an architect I am definitely better off when developers show me where I was plain wrong. Believe me, this is much better than testers, reviewers or customers happily finger pointing to your design flaws. Thus, architecture enforcement is a bidirectional interaction between architects and implementation, with architects being the drivers.

As Lenin once said: trust is good but control is better. Think about this before throwing your design over the fence in the next project. Some things tend to backfire heavily.

The Extensibility Syndrome

In one of the Facebook groups I am member of there is currently a discussion going on how to deal with extensible design, especially how to prove to a client that a system meets its extensibility expectations.

Focus on extensibility is important but can also be a big hazard, in particular, when engineers overdo it. For example, consider the application of the Strategy pattern. Basically, applying the Strategy patterns implies some kind of uncertainty which algorithm actually should be used. If many Strategy pattern instances are used, this almost always means that the engineers have no clue in which direction the system should be extensible. Of course, the same holds for overuse of other extensibility patterns as well. Such systems become a nightmare for developers and users.

Thus, make clear with requirements engineering or customers what kind of extensibility they really need.

What needs to be extended:

• Should an algorithm be exchanged?
• Is it a whole subsystem or layer that is subject to change or extensibility?
• Is it a component that needs to be changed or extended?

When will it be extended?

• Now,
• Mandatory in the future,
• Optionally in the future?

Which binding time?

• source code,
• compile time,
• link time,
• runtime,
• maintenance time?

Who will be in charge of extending?

• operator,
• user,
• developer

You should only consider those extensions that are mandatory, none of the possible or optional ones that may or may not appear in the future.

In the beginning of software design introduce change scenarios as proposed by the book Software Architecture in Practice, 2nd edition (Bass, Clements, Kazman). Here you’ll learn how to describe modifiability scenarios using scenario diagrams and rating them using so-called utility trees.

You may also use design tactics diagrams that deal with how to implement the different kinds of extensibility mentioned above.

If you follow the principles of my Onion Model, then each architecture design step will only be driven by requirements and their priorities. This way, you can establish requirements traceability within your architectural design.

For extensibility and change scenarios, a Commonality/Variability analysis can be leveraged, especially when dealing with a wide variety of extensions such as in platform or software product line development.

In contrast to common believe extensibility or change is only applicable after you’ve designed those parts of your architecture that are subject to extension or change. You can either extend some functional entities or operational qualities such as scalability mechanisms. This implies that all extensibility and change design activities follow after functional and operational design and not before.

When opening your functional & operational design for extensions or change always use the Open/Close principle to prevent unwanted side effects.

To prove that a software architecture is extensible such as expected by the customer you got different means.

• An end-user or operator might like to see the implemented change scenarios and how they actually work in the implementation.
• Another engineer might like to see how the architecture and implementation support extensions.
• A requirements engineer may like to see the mapping from requirements to architecture decisions.

Modifiability aspects (e.g., extensibility and changeability) are much more difficult than they seem at first place. When the amount of extensibility is overwhelming, you’ll often find wizards or configuration automatisms to hide the mess underneath (which does not imply that wizards or configurators are indicators for bad design).

As Heraklitus once said: panta rhei – everything changes. Thus, be prepared!

Does Cloud Computing have an Architectural Impact?

Let’s suppose you have good reasons to leverage a Cloud infrastructure. Will this decision hava any impact on your architecture? The answer is quite easy: It depends :-)

• If you just use virtualization to exactly simulate a physical environment like in Amazon EC2, then your applications will be oblivious to the Cloud infrastructure.
• If you leverage SaaS (Software as a Service) like Salesforce.com then it’s a SEP (Somebody Else’s Problem).

However, if you use PaaS (Platform as a Service) or IaaS (Infrastructure as a Service) for your applications, then things definitely will change.

It is not so much the domain-specific logic that is influenced, but infrastructural and non-functional requirements will experience an impact.

• In a Cloud environment different storage means are supported. In general, you’ll find Blobs and NoSQL databases. Designing the persistence infrastructure must take these issues into account unless you are still relying on a mainstream DBMS running somewhere in the Cloud.
• For messaging Message Queues are available. Thus, message-oriented communication between Cloud-aware subsystems is a reasonable approach for connecting islands with each other. Often, SOA style communication protocols such as RESTful communication or SOAP/WS-* are supported. These protocols alone imply different architecture paradigms - designing the communication and distribution infrastructure can be a challenge.
• All operational qualities such as performance, availability, scalability, fault-tolerance need to be implemented with the SLAs in mind the Cloud provider offers. If, for example, availability zones are supported in the Cloud you better plan how to leverage them according to your needs. In a public cloud security is even a more critical issue. You cannot naively trust the Cloud provider to keep your mission-critical data secret. Thus, you have to provide means like encryption almost everywhere. For performance reasons, you might want to consider the Cloud as a Grid such as in Apache HADOOP. But how can you partition your system in a suitable way. Is Mapreduce what you need or, otherwise, does your application require a different kind of approach? There are many different concurrency architectures like Master/Slave, Pipes & Filters, Concurrent Reactor that could be appropriate. In contrast to typical application development you are much more constrained when addressing these operational qualities, because many decisions the Clound infrastructure has already made for you. It is YOU who needs to integrate with the Cloud, not vice-versa.
• Developmental qualities such as modularity, modifiability, maintainability can be hard to achieve in such an environment. First of all, you should introduce some kind of governance approach. Secondly, you need to modularize in such a way that governance issues live in harmony with the kind of modularization propagated by the Cloud, in particular by the PaaS APIs. For command & control purposes you need to think about how to administer the Cloud applications. There might be 3rd party products for this purpose. It, however, might also be the case you have to implement your own tooling.
• If, for some reason, you need to use different cloud infrastructures, interoperability might be a big issue given that today’s approaches almost inevitably lead to vendor-lock-in.

Not always, you will be able to start with a green field project, but rather need to deploy an existing application ecosystem or parts of it to the Cloud infrastructure. Migration is not straightforward as we already saw earlier. You may need to combine reengineering and refactoring activities for this purpose.

This posting describes only the tip of the iceberg. I hope, people will develop some best practices (patterns) for how to leverage existing Cloud platforms. I’d be rather sceptical if someone would come up with such patterns soon. You surely remember, patterns must be independently used in three different applications to make them patterns. I cannot imagine, we already obtained that much experience in practice.

So, if you are going to design for Amazon EC2, Google Web Engine or Microsoft Azure don’t forget: Mind the Cloud!

Requirements traceability – The Holy Grail

It often occurs that I am supposed to review an architecture document – that’s one important part of my job. After having read the document, I always try to summarize what I just read. And in many cases, this proves to be rather difficult. One of the reasons is that the subsequent parts of the document seem to be only loosely connected such as graphical models popping up without textual explanation or architectural & technical decisions appearing without any hint what the design rationale could be. However, a reader who is dependent on rumors, insider knowledge, and vague assumptions will see the hodgepodge of syntax but never fully understand the semantics behind this syntax. If the reader is a tester, requirements engineer, fellow architect, or developer such situations need to be considered harmful. Yes I can see the answer is 42, but was the question respectively problem that lead to this result?

The big secret is requirements traceability. Each and every architectural decision must be strongly derived from forces (i.e., requirements and risks). This also helps to keep the architecture simple, expressive and balanced, because this way we can get rid of all these design pearls smart developers and architects typically invent.

From the forces elicitated at the beginning of a project architects derive architecturally relevant requirements. This includes requirements clarification and priorization. Remember: Garbage-in-Garbage-out. If the initial assumptions are wrong, the resulting implementation can under no circumstances meet the expectations.  Of course, we do not expect requirements to be fully available at project startup. We have become extraordinary agile wizards, anyway, and can also handle requirements dropping in. Needless to say, we can not gurantee the same perfect software & system architecture we would have come up with if all these requirements had been fully available from day 1.

As we already got those long-term experience, we are using the onion model.

• For all use cases (part of user requirements) we are deriving a black box view of our system. From this descriptions we also obtain actors and required/provided interfaces. As we see, this step is driven by requirements.
• The domain model is another force. It is a means to understand typical entities in the underlying domain as well as their relationships and interactions. They will turn the black-box view into a white box view. Now you can start to map the use cases to sequence diagrams which is also requirements driven. All of a sudden the domain-specific subsystems, components, interactions, interfaces are disclosed refining our external model. Eric Evans is an excellent source how to deal with such domain-driven design.
• So far we have focused on the domain entities. It is time to add infrastructural requirements such as communication, persistence and concurrency infrastructures as well as operational requirements such as performance, safety, availability. This we do by starting with the highest priorities- that’s why priorization is so essential – and ending with the lowest priorities. Each architecture decision is strongly rooted in requirements.
• The same is done with the mandatory developmental requirements like modifiability, maintainability and then with the optional developmental requirements.

All these decisions are depicted in diagrams, but also explained in the textual descriptions.

What are the benefits of such an approach:

• We get requirements traceability: Firstly, we know which requirements lead to which architectural decisions, thus also obtaining a mechanism to check whether we have already covered all the necessary requirements in our design. Secondly, we know which effects an architectural change such as a refactoring will have on requirements, thus preventing or at least lowering the risk of accidental complexity. Thirdly, all decisions are made explicit. Implicit parts lurk like invisble ghosts deep in the guts of the architecture hurting us when we are least prepared.
• We are driven by priorities. If we miss our milestones, we skip the less important requirements instead of the really important things.
• Readers of our documents will not only understand the “what” but also the “why” making it much easier to get their buy-in. This is what we call design rationale.
• If we additionally come up with leading guidelines and principles for cross-cutting concerns in the beginning we will increase internal qualities such as symmetry, orthogonality, simplicity.

The journey is the reward in software architecture. Don’t tell readers about the target to aim for, but also about the way how you get there. On each junction, explain why you prefer one over the other road.

Writing of such documents can be such a fun. And reading them even more. For example, read Dave Cutler’s book about the creation of Windows NT.

A architecture document shall contain guidelines and design rationale, but no labyrinths.

Back from the OOP 2010

This year’s OOP conference in Munich has been a lot of fun. I met Markus Völter, Stefan Tilkov, Floyd Marinescu, Jan Bosch, Eric Evans, Philippe Kruchten, Roman Pichler, Peter Hruschka, Kersten Auel, Rene Schönfeldt,  Klaus Rohe, Matthias Bohlen, Gernot Starke, Jutta Eckstein, Nico Josuttis, Gregor Hohpe, Robert C. Martin, Matthias Bohlen, Kevlin Henney and my good ole friend Frank Buschmann. Especially the networking part was overwhelming. So many speakers and participants you can learn from.The talks and keynotes are definitely helpful to hear about new trends, perspectives and experiences. But the personal communication is the essence of such conferences. That’s the reason why I don’t believe in online events. Sure, it’s a nice add-on but it cannot substitute the “human” factor. In the middle of the week we had a meeting of the Siemens Senior Software Architects where I enjoyed meeting some of the aforementioned celebrities but also the smart senior architects within Siemens who do excellent work day by day. They might not be that much in the limelight, but those people are  the real heroes of software engineering among all those “nameless” other heroes such as development heads, team leads, testers or developers. As Jan mentioned: there are two types of architects. Those talking about software architecture and those practicing it. Fortunately, I am a hybrid belonging to both types.

This year’s keynotes  offered excellent quality. I’ll never forget Uncle Bob’s entertaining talk on the polyglot programmer where he dived into the history of programming languages in a very entertaining way. Or two speakers from Zühlke Engineering who presented a great keynote on functional programming. Not to forget Gernot Starke with his talk on software architects and stealing in your neighbor’s garden. Unfortunately, I could not make it to his keynote but from what I heard it was one of the highlights.

Most of the topics presented in talks and tutorials covered the typical topics you’d expect. Cloud Computing and Functional Programming as well as Agility represented the hot topics. What I liked most were all these more practice-oriented talks offering some insights for practioners and also revealing some real life show cases. For instance, I enjoyed Kurt Höpli’s talk on building and controlling environmental sensors in a project in Switzerland. And I also experienced a lot of fun in Markus Völter’s talk on how to apply Model-Driven-Software-Development  within an embedded systems context. Never I will forget how Eric Evans and Hans Dockter presented Domain-Driven-Design convincing some attendees to participate as actors. And I’ll remember also Lothar Mieske who presented the real look-and-feel of all those Cloud Computing platforms. I could add much more but this should give you an impression and suffice as a pars pro toto.

Personally, I have been very busy the week. First, I gave a full-day tutorial on Software Architecture – From Requirements to Architecture. It was the tutorial with the largest numbers of attendees. This makes interaction somewhat difficult but nonetheless worked much better than expected. My talk on Hitchhiker’s Guide to Cloud Computing had about 80-100 attendees. To my pleasure the Global Technology Field Cluster Lead of my organization also attended the talk. For me, it was the biggest surprise that the talk “Introduction to Scala” had attracted so many people. I was totally electrified and I mean this literally cos’ static charge hit me whenever I pressed the page-down key. When people are interested, I will record this talk with Camtasia and publish the video clip in YouTube and other media. All in all, I got really overwhelming feedback. Thank you to all attendees for their nice evaluation sheets :-)

Next year the OOP will celebrate its 20th anniversary. Hope that many of you will join the event. It is really an extraordinary experience.

Stakeholder-specific Communication

Communication is one of the most importants aspects of software architecture design. The architecture itself is a means of communicating design decisions to customers, product managers, project managers, testers, integrators, and developers. However, these stakeholders have different expectations regarding the software architecture. While developers need a clear guideline how to implement the architecture design, customers may be more interested how (well) the software meets their requirements. The software architecture documentation and all other means of communication such as presentations must cover these various perspectives. To address such broad spectrum of interests, architects could provide a specific document for each particular kind of stakeholder which leads to an unacceptable amount of effort and time. This is why documents should rather explain the architecture decisions - the "what", the "how" and the "why" - in a top-down approach and add a guide-to-the-reader motivating who (i.e. which stakeholder) should read what in which order. For presentations or other forms of communication it might be worthwhile to introduce a specific variant for each stakeholder. A project manager or customer is typically not that interested in all those strategic and tactical design destails, while developers need all the information they can get. Thus, communication between architects and other stakeholders should take the stakeholder-specific interests into account. This implies, that architects need to have a clear understanding of these interests before starting documenting or communicating. Otherwise, they won't get buy-in for their software architecture by these different stakeholders. Personally, I have experienced so many projects where managers couldn't follow and understand what the architects told them overwhelmed by details they did not (have to) care about. And I also have been in so many meetings where software architects introduced high-level abstractions that could not provide the details developers and testers expected.
As a prerequisite it is necessary to ensure a common understanding of terminology. Did you ever attend a meeting where different participants had a different understanding of terms? Communication will not work and succeed when one person is speaking english, while the communication's target can only understand spanish. This might be obvious for real languages, but it is often underestimated for domain-specific languages. Did you once try to read and understand (!) a treatment record of your physician? He speaks english, you speak english, but this won't help you much. As a consequence, introducing a domain-specific language is valuable and essential. It does not have to be a full-blown language in the strict formal sense, but often it makes sense to offer more than just a glossary. The core functional architecture should represent the problem domain.
As a consequence, all communication between architects and stakeholders must happen in a stakeholder-specific way. When giving a presentation on the software architecture, make sure you know the target audience and their interests. Don't assume, other types of stakeholders understand software architecture the way you do. You are the doctor and they are the patients, so-to-speak. Communication may succeed or fail. The risk of failure can only be minimized if you can take and understand the perspective of your communication partners.
Obviously, these statements also hold for other kinds of communication. Have you ever tried to explain to your spouse why you need that cool new gadget for several hundred bucks? Try that by telling her/him the technical specifications. Won't work very well, right? What we take for granted in such situations (i.e., adapting the communication to your communication partner), represents also a conditio-sine-qua-non in software engineering. So, better mind the stakeholder-specific communication in the future :-)

Real Software Engineers

• Real software engineers don’t need safety nets. Instead, they are ready to face all dangers and risks without any fear. For this reason, they dislike (unit) testing, because (unit) testing is for pussies (i.e., those who do not trust their own code.) Since real software engineers never make mistakes, testing would be a mere waste of time.
• Real software engineers won’t document their design, because these designs contain an incredible amount of design pearls other mortals would not be able to comprehend, anyway.
• Nor do real software engineers read other design documents nor manuals, either because their fellow engineers did not provide documentation or because they do not trust these documents.
• Instead, their main guidelines are:
• All you need is code
• Bubbles don’t crash
• Real software engineers do not check or prioritize requirements. Instead, they got intuition for customers’ needs which drives all their activities.
• Real Software Engineers do not communicate (much) with other stakeholders, because other stakeholders are just obstacles on the way to project completion. That’s also the reason why real software engineers hate meetings.
• Real software engineers do rarely speak English, German, …. They rather express themselves using C#, Java, C++, …
• Real Software Engineers feel more comfortable in the solution domain. Their goal is to provide solutions instead of getting lost in problem domains. This is why they prefer playing around with solution technologies.
• Real software engineers can turn any problem into a nail for which they can use their favourite hammer.
• Real software engineers do not need software architecture because a predefined architecture would just limit their creativity. A good architecture is always created ad-hoc for solving real problems in a pragmatic way.
• Real software engineers do not foster re-use. First of all, re-use has never worked in practice. And secondly, re-using design or code typically requires more resources than reinventing the wheel. This is often due to the fact that re-usable artifacte were created by other real software engineers who failed to document their work.
• Real software engineers won’t read this blog because in the time needed for reading, they could add another feature to their software.