A picture can say more than thousand words - a word can say more that thousand pictures

One of the key problems in architecture documents is the idea illustrations are sufficient for explaining design.

Let me give you an example:

Eo---oP

What could the diagram illustrate?

Could this describe a hydrogen atom consisting of a proton and an electron?

Does it represent the association between an entity bean and its persistence store?

Or is it something completely different?

Diagrams are models. They can only express a subset of reality. If you do not know the concrete context, you’ll have no way to figure out what the diagram actually means.  But if you are familiar with s special context, you might be inclined to map a diagram to your own context which, however, might be painfully wrong.

Thus, an architect should always explain the context of a diagram as well as the constituents and their meaning textually in the document.

What about the other way round?

Again, an example:

Component A is connected with B, C, D. D is connected with E and C. F is connected with D and A … a lot of more information …

In the example above, plain text is not really helpful. Here seems to be a good place to add a diagram that graphically illustrates the components and their connections. Text alone is not capable of describing complex structures which often appear in architecture design.

Hence, we recognize that diagrams need explanations to understand their semantics. Otherwise they just represent syntactic glibberish. In addition, UML, SysML and other graphical notations might lead to ambiguous illustrations which are open to interpretation. Thus, tell the reader what exactly the model behind the diagram is supposed to be.

As a consequence, all design documents that are automatically generated have almost no value.  For example, design rationale (such as requirements and their association with architectural decisions) is better described using textual language instead of diagrams.

Thus, use diagrams carefully in your design documents. But also be aware of text that does only describe what is depicted in diagrams, but does not introduce information such as pre- and postconditions, invariants, contextual information, design rationale, …

Guess, why there are UML use case diagrams that illustrate a set of use cases, but no graphical means for describing the use cases themself?

Use text and diagrams wisely! Don’t consider design documents as write-only-entities. You will see what I mean whenever you have to understand  documents of your fellow engineers.

Maturity matters

This time a rather small posting only.

I am wondering what the top research fields in software architecture currently are? Of course, fields such as MDSDL/DSLs come immediately to my mind. Cool stuff! Investigate these topics and fame is guaranteed!

Others are already sufficiently mature. I am thinking of mapping requirements to architecture decisions. Ok, maybe there are still some deficiencies in this area. Hmmh, there might be even significant gaps, especially when dealing with operational and developmental qualities. Must reevaluate my opinion.

What about documenting software architecture? There are dozens of templates available, even books. A lot of agreement here, but what about modeling? Is UML the “ultimate ratio”? How can we model, what should we model? How can we cover design rationale? How deep should we go? Did I ever read a really good architecture documentation?

I got it. Patterns are super mature. I have been author of pattern books myself. These books are now available since 14 years. Some architects even know the names of all patterns. But do most architects know patterns others than GoF? And even if they do, do they apply them?

Isn’t refactoring a good candidate or reverse engineering? Architecture refactoring isn’t established in most development organizations. If I say architecture refactoring, I really mean architecture refactoring, not code refactoring.

What about software architecture design? We have been designing software systems for decades and they work! No, most of them suck! Unfortunately, design by accident is the norm, not the exception.

… thinking, diving deeper, diving broader, getting frustrated …

Why is my impression that most areas in software architecture are still rather immature?

Maybe, the answer is that software architecture itself is a big collection of research topics.

Or maybe, I am blinded by the light.

Architect – what architect?

In many projects and organizations I am constantly meeting software architects. Given the quantity I’d only expect only a fraction of software development projects to encounter any issues in software architecture and design. As we all certainly know, quite the opposite is true.

Ok, nice and true words, but what exactly is the reason for this observation?  

Of course, I absolutely have no clue :-; So let me just make some assumptions.

Could it be that the term “software architect” on a business card is subject to a variety of different interpretations?

In one company, someone may be promoted to software architect, just because she/he is the only one who owns a pattern book. [Beware of humour!]

Consulting companies may entitle some of their engineers as software architects, because they can charge more for architects than for developers.

A handful of companies may offer a challenging education program to become a software architect.

With other words, it is not clearly defined what a software architect is or should be. Nothing prevents you from calling yourself a software architect. You may remember that I am proposing a rather tough profile for software architects. Well, I know too many projects that failed  miserably because of lack of competencies, experience or skills by the responsible software architects.

Note: not only the skills count! As software engineering is heavily related to communication and learning from failure, becoming a software architect requires starting as a developer, taking over more and more responsibilities over the years, before finally turning into a butterfly.

Another issue I often encounter is the missing separation between software and system architects. Today, many system architects are also considered as software architects. This might work very well if the system architect is an experienced software architect. Likewise, the aforementioned setting is adequate for software-intensive developments. But here again, the system architect needs also to be an experienced software architect. If the latter doesn’t hold, the software relevance is obviously underestimated and will inevitably lead to trouble.

Hold on, there is even another possibility. Even if you can justify that you are a software architect now, this doesn’t mean you’ll be a software architect forever. One of the most fundamental challenges is to know the state of the art of relevant software technologies and methods as well as appreciating and practicing coding and testing skills. Guess, you have been appointed as a software architect ten years ago, but in the meantime you were constrained to using UML tools, Powerpoint and other Office applications. With other words: would you trust a surgeon to operate on your heart who lacks ten years of practice?

I bet, you made some similar experiences in your environment. If that is the case, a comment would be highly appreciated. If not, even more! 

Sick Architecture

You may wonder why my publication rate in this blog has decreased dramatically. I am staying at home due to an infection for three weeks now which wasn’t that helpful for my blog.

To keep things running I have decided to create a new posting. Well, the title is intentionally related to my current health situation.

To be honest, I think that software architecture design reveals some interesting commonalities with sickness:

• Both often require intense treatment by experts
• Different experts offer different opinions and treatment plans
• Both are typically open-ended to some extent
• Both are not subject to planning in most situations
• Both mostly feel very miserable
• To reduce the risk of getting infected some kind of prevention strategy is always a good advise
• It is better to cure the cause not the symptoms. Thus, be aware of all interdependencies and follow a holistic approach
• Viruses are a problem in both worlds
• Healthcare never ends – it is important in the whole lifecyle
• Learning from failure is recommended to avoid similar problems in the future
• Prevention is always better  (i.e, cheaper) than healing

Most software architectures I know are not in a good and healthy shape - at least in some intermediate phases. Why? Because they were not devleoped with the aforementioned bullet list in mind.  Thus, health monitoring (i.e, reviews) and treatments (i.e., refactoring) should be an essential tool in the architect’s framework.

Does this sound reasonable?

Data centric Architecture

In most projects there is a big emphasize on functionality. Use case modelling defines responsibilities of the system as well as how it interacts with actors. Logical modelling then maps the use cases to related subsystems/components which, of course, is functionality based. At the end we'll get functional interfaces between subsystems or between components. Do you recognize that methods such as UML mostly focus on methods?



However, in many cases the data exchanged between architectural entities is rather complex. For example, in healthcare scenarios complex structures with patient information or charging information are the rule. Not dealing with data as first class citizens might lead to various issues such as versioning problems, or inadequate partitioning or aggregation of data. Thus, architects should deal with data modelling very soon if they are facing complex structures in their development projects.



Data modelling starts with the domain model. What are the main entities in your problem domain and which kind of data do they need to persist, share, or transfer?

Then in the next "layer" when we are introducing the subsystems we'll have to think about data structures in subsystems and data structures exchanged between subsystems. For example, what exactly are the data structures required in the functional interfaces. Obviously, these data structures can be partially derived from the domain model with further data structures required by infrastructural entities or non-functional qualities.

In the component layer we are starting to deal with finer grained data types.

Last but not least we'll have to define how data is mapped to implementation artifacts.



Thus, we should also follow a top-down approach for data modelling. And likewise we need to consider bottom-up constraints such as special data requirements of infrastructures or platforms. With other words, data modelling must go hand in hand with all the other architectural modelling activities:



Important questions to ask when modelling data:

• Who is the originator of data?


• Who is the consumer of data?


• How does data move from the originator to the consumers?


• Do we need to collect/aggregate data from different sources (transfer objects)?


• What about quality constraints such as performance (e.g., caching data), security (e.g., encrypting data), modifiability (e.g., versioning data), concurrency (e.g., locking data), availability and reliability (e.g., replicating data, data persistence)?


• Where and how is data stored?


• How do different data structures depend/relate?

I think, data modelling is an underestimated and undervalued activity in architecture design. What about your experiences and opinions?

Ineffective Communication

It might not be that obvious why time management is of high importance especially for software architects. But of course, being a software architect in a mission critical project requires your full attention. However, in a couple of projects I met software architects who were simply overwhelmed by communication requests and finally got stuck in a kind of waste time singularity. Ineffective communication can significantly reduce your productivity. 

Unfortunately, there are many opportunities where your focus on the real important things is easily lost.

• My favourite are unnecessary meetings (including telephone conferences).  As software architects are caught in the gravitational center of the project they receive a lot of meeting requests, by managers, developers, other engineers. If you are a very communicative and friendly person – and I bet you are – you might be inclined to say “yes, we can” to all those nice invitations. It is often overseen that meetings also require preprocessing and postprocessing activities. Hence, don’t just count the time you’re spending in the meeting room! Consider carefully what type of meeting you are asked to attend. Is it an essential meeting where your participation is absolutely required (class A meeting)? Or more one of those nice-to-have-meetings? Only accept class A meeting invitations! Even for class A meetings ask yourself whether this meeting has the right setting in terms of people invited, time frame, goals, agenda. With other words: is the benefit you obtain larger than the time spent? Sometimes, a small face-to-face meeting offers more value than a large official meeting. And of course, act thoughtfully if you invite to meetings yourself! As an Outlook junkie do not only enter meetings in your calendar but also time slots for your actual work. Mind the fact that other people tend to fill the gaps in your calendar assuming those gaps are indications of idle time. Personally, I also enter slots for my spare time, just in case. Needless to say that you should also consider travel time in this context. Care about meeting netiquette: there is no place for cell phones or notebooks in a meeting – a few exceptions granted.
• Another critical challenge is e-mail. Communication by e-mail can be very ineffective (but also the most effective means in other situations). Try to reduce corporate spam which consists of all those mails which steal your time for no good reason. Organize your inbox in folders and prioritize mails as soon as it arrives. Process the mail depending on its priority. Read mail only at specific points of time and tell your communication peers about this fact. And do not send unnecessary mail yourself! Before writing an e-mail, always hold on and consider whether you would also have sent an letter containing the same information in the stone age of communication.

At the end, it is the project success that counts. Good results require effective architects with as much productive working time as possible. Productive hours depend on effective communication. It is the quality not the quantity!

Vacation - the end is near

The last weeks I have relaxed a lot: visiting Dresden, running, swimming and biking, as well as practicing with my DSLR. In addition, I read two books on Scala, updated an article on D (the language), installed several apps on my IPhone (yag – yet another gadget),  prepared slides for ADC 2009 and OOP 2010, participated in a new episode for our SoftwareArchitecTOUR podcast, … and spent a little time on further expanding my work on architecture refactoring.

However, I intentionally did not work on software architecture or on this blog. This will change beginning with upcoming monday. Now, that I got some new and fresh ideas, I can continue my journey on software architecture issues.

By the way: I did not mention it so far but maybe you like to follow me on twitter (username: stal_m).

And now for something completely different

This is my first week of a 3 weeks vacation. It is really hot in Munich why I am currently enjoying swimming, biking and visiting beer gardens more than software engineering.

Nonetheless, I am currently educating myself in a new programming language. Scala by Martin Odersky combines a pure object-oriented langauge with functional features. And it runs on the JVM. An earlier version of Scala does also support the .NET CLR.

What I like most is the proper integration of functional features. Closures, for comprehensions, nested functions, pattern matching are notable examples. While Scala is statically typed (it is not a dynamic language), it offers type inference for variables, arguments and results.

In addition, it provides some cool extensions to Java: all types are objects, no static members, support of covariance and contravariance, to name a few.  Traits are available instead of interfaces – they can behave like interfaces but also allow implementations of methods. With traits the problem of multiple inheritance (mind the diamond configuration) can be avoided. This is possible by introducing inheritance linearization and a kind of virtual super.

With all of these features Scala code can be much more compact than Java - and it interoperates well with Java components.

Well, I am really kinda enthusiastic. If you ask me, Scala could be a strong competitor for Java and C#.  But of course, there is not a big company backing Scala such as Sun or Microsoft. Thus, the community will decide.

Loving the JVM but preferring other languages than Java? No more excuses, Dave!

Neverending Story

One of the prominent questions I often hear in software development projects is when to start with architecture design. Sometimes, architects hesitate to start until almost all requirements are available in high quality. This strategy is highly recommendable when you like to postpone architecture design forever. There is not a single project with medium to high complexity I remember where we knew all forces in full depth and breadth at project start.

First of all, it heavily depends on the software development process. In a Waterfall model without iterations you are stuck anyway. Thus, I generally recommend an iterative-incremental approach. In software developments with all their volatility only piecemeal growth can succeed.

It is important to have a sufficient quality and quantity of requirements available, for example, the most important 30%-50% of use cases and non functional qualities with unique priorities. This way, you can come up with a conceptual architecture which you can then use as a base for iterative-incremental refeinement.

After the baseline is available prioritize all user requirements (use cases), map them to end-to-end scenarios and integrate them into the architecture, keeping the non-functional qualities in mind. To deal with non functional qualities let key developers, architects or even teams work on principles and guidelines how to address these qualities. Do this after the baseline architecture design but before starting the end-to-end iterations.

As soon as new requirements appear or have been concretized assign them priorities and put them into the “backlog”.

To introduce a kind of safety net for embracing change, introduce test-driven design methods, flash architecture reviews after each iteration, and (architecture and code) refactoring activities after each flash review.

Keep in mind that sometimes you need courage to decide even when not all details are available.  It is typical for all creative and volative acts that not all details are known in advance, nor the ways how to cope with them.

As a consequence, it is much more productive and effective to follow wrong paths sometimes instead of following no path at all.

To keep it simple: Prefer acting over reacting

May the force be with you

Sometimes it is getting confusing. There are so many influential factors driving software architecture. Here is my small domain language

• Forces is the set of all influential factors an architect is facing: requirements, risks, constraints, business needs.
• Requirements are the (documented) set of all expectations for a software system. According to Wikipedia a requirement represents a necessary attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user.
• System requirements address the system as a whole. From system requirements we need to derive architecturally relevant requirements that drive architecture and design. That is the architect's job.
  
• A Functional requirement defines functionality a system is supposed to provide, sometimes also known as capability. Use cases are functional requirements. They define what the user expects from the system on a black-box level. User stories according to Wikipedia denote software system requirements formulated as one or two sentences in the everyday or business language of the user.
• A Non-functional requirement (also known as quality requirement, quality attribute or "ility") is constraining a solution. There are two subcategories here: operational requirements as the name suggests constrain the runtime behavior of the system (e.g., security, performance) while developmental requirements constrain the design from a developmental perspective (e.g., maintainability, modifiability). As operational requirements often have a systemic approach to a system they are subject to architecture design and therefore are also called strategic requirements. Tactical requirements such as modifiability mostly have only local scope.
• Constraints - what a surprise - are constraining the solution. A constraint might be technical (we need to use Windows 7 as our OS), organizational (subsystem A should be developed at location A), standard & law (we must abilde to FDA rules), business-related (target costs should not exceed xxx$).

It is the job of a software architect to base all decisions on the forces of the project. Still she/he has sufficient freedom to make architecture and design a complex task. Architecture is about communication and courage to decide.

No future for architects?

I am a big fan of Star Trek, especially of The Original Series and The Next Generation. From a technology viewpoint, it is amazing how all of these technologies such as warp drives or beaming have influenced science. Did you know that the inventor of the mobile phone had also Star Trek in mind?

However, there is one sad aspect. What about Software Engineering? Whenever software plays a role in any of the episodes, you’ll recognize Spock or Data just coding. Do they ever design? There is also no software engineering team in the Enterprise although software should be an important asset in the system architecture. I am missing a chief engineer for software engineering aspects!

Does the computer program itself? Will UML disappear in the future? Or is it just software engineering being too boring for SciFi fans? Interestingly, there are also seem to be no restrooms in the spaceships. This is what they have in common with software engineers :-)

Thus, we could ask how software engineering especially the discipline of software architects will evolve in the future? Will we still use a successor of UML in hundred years? How could the role of software architects evolve?

The best approach to predict the future is identifying space of improvement. It is very likely that such areas will be addressed. How should the ideal process of software design look like? What is the software architect supposed to know and to do?

In most SciFi movies automatization/Model-Driven Software Development and AI are often the answer. Architects express their intent, smart systems then try clarify open issues, and eventually the result is generated.

Another approach could be an organization like the Borgs who strongly follow an agile approach with code ownership, collective programming and test-first.

Any opinions?

Keep in mind: Resistance is futile!

RRC Card

Most of you already know the CRC (depicting Class, Responsibilities, Collaborators) which is a low tech technique for discussing software artifacts in brainstorming sessions for software design.

I propose another kind of card for depicting the roles in a particular project: the RRC Card: Role Responsibilities Collaborators.

The idea is simple:

• The first R stands for role by which I mean a concrete person and her/his role
• The second R stands for Responsibilities: what responsibilities should this role have to meet its expectations
• The C stands for Collaborators: from which other roles does this role depend, e.g., with which other roles must it interact

If you start with a project, you could ask the parties involved to jointly fill out these RRC cards, thus making clear how responsibilities are partitioned and how roles are supposed to interact.

Example:

---------------------------------------------------

R: Michael / Architecture Consultant

---------------------------------------------------

Responsibilities                        Collaborators

Mentor and coach                    Lead Architect: Paul

Help design                             Architect: Tom

Implement

----------------------------------------------------

 

If you already in a project in deep trouble you could ask persons to fill out how they live their own role and likewise fill out RRC cards for other persons, thus identifying conflicts between expectation and reality.

Document these RRC cards in a document (e.g., paper-based or Wiki-based)

Risk-driven

In addition to requirements and business expectations, risks represent an important driver of architecture design activities. Not addressing risks appropriately is a major reason for project failure. But what exactly is a risk and how can I identify potential risks as an architect.

On a very coarse grained level, risk might be considered as not knowing the outcome of an event or activity. It is the threat or propability that “an action or event will adversely or beneficially affect an organisation's ability to achieve its objectives” [Wikipedia].

There are as many different types of risks in a  software development project as there are influential factors.

For example:

• Technical risk: Is EJB the right technology for achieving 100000 transactions per hour in our Web store?
• Organizational risk: Is the organization appropriate for this kind of development project?
• Political risk: Does this product manager follow her/his hidden agenda? Is this supplier really trying to support us or is he in fact competing with us?
• Design Risk: Is this design decision appropriate for dealing with the requirements?
• Requirements risk: Do we really understand the requirements of our organization or the customers?
• Process risk: Is this the right development process?
• …

For dealing with risks, there are lot of different ways. Technical risks and architecture risks can be reduced by a requirements-driven, test-driven approach with explicit assessment activities and refactoring activities. Political risks require the architect to communicate and lead efficiently and effectively.

An important point is this context: to be able to deal with risks, we must make them explicit. Implicit risks are often overseen, forgotten or ignored. You should really know your “enemy”.

Making risks explicit means to document AND communicate them.

I recommend to introduce a template for risk assessment comprising:

• Name of the risk
• Priority/Relevance of the risk
• Type of risk: organizational, technical, ….
• Rationale: explain why this is a risk
• Context: what else should we know when dealing with this risk
• Strategies: possible ways to deal with the risk
• Decision: the recommended strategy to deal with the risk for the concrete context, and the rationale for this decision
• See also: other risks this risk implies or similar risks that could be treated with the same strategy

Not all of these fields need to be available in the beginning. You could start with name, type, context, thus collecting the risks first. Then, you could prioritize the risks, and eventually extend the templates for this risks identified as high risks with missing information.

Let me give you an example:

• Name: Uncertainty Using EJB
• Priority: high (******)
• Type: technical
• Rationale: capability to meet the performance requirements of 100000 transactions/hour. We are not sure whether EJB will be appropriate
• Context: Our developers are not experienced with EJB
• Strategies:1) Build performance prototype first. 2) Hire external expert. 3) Use Enterprise OSGi instead. 4) Outsourcing.
• Decision: Hire external consultants. Let them build a performance prototype and then help implementing the system when EJB proves to be a reasonable choice.
• See also: we are not sure whether MySQL can cope with the throughput requirements.

It is worth mentioning that risk assessment is not exclusively provided by software architects. In fact, all roles in the project must help and closely cooperate for risk assessment. This way, everyone involved is aware of potential risks. And of course, some risks such as those implied by organization, test quality, or quality of requirements must be identified and clarified by other roles such as product and project management, requirements engineering or test and quality management.

Architecture and design decisions should then take risks into account.

Mind the risks: The highest risk is not systematically dealing with risks. 

Who is responsible

Suppose, you are working in a software development project as a software architect. You are still waiting for the requirements to be available in sufficient quality and quantity and priorization. Even after months almost nothing happens. Sure, the problem is caused by requirements engineering. But what is your responsibility as an architect? Your responsibility is to communicate and escalate which requires courage. For example, you could tell the stakeholders what implications the current situation might have - for example, you cannot start with architecture design which means development cannot start with implementation etc. You should never fall into the "this is not my job" trap. After all, software architects need to drive the project. Sometimes, it might be even possible to take responsibility for requirements as an architect but this should be the exception not the rule.
Similar issue for test & quality: if test coverage is only limited, bugs are seldomly found during testing, or tests themselves are faulty, then you could finger point to testers and developers. However, it is your job as an architect to deal with quality. Don't try to escape your responsibility in this area. Test First Design makes it very clear that testing requires the different roles to cooperate.
What about business aspects? I know, you got this product manager who is in charge of all business aspects. But software architects are responsible in helping develop the technology roadmap, identifying patents, estimating the business implication of architecture decisions. Even more, architects must base every architectural decision on requirements and business needs.
Hmmh, what about project management? Of course, an architect should not be forced to also act as a project manager. These two rules are very difficult to live at the same time in the same project. But what if project management needs to estimate costs and resources? Here the software architect needs to provide support. What skills do developers need? How many developers and how much time are necessary to develop subsystem A.
I don't claim that architects shuld be jack of all trades but masters of none. Their main focus should be on software architecture. However, they cannot ignore what is happening on the boundaries between software architecture and the rest of the project. They should feel responsible for project success.

Domain Models (continued)

Throughout my life as a software engineer I have participated in many different projects in various domains. Needless to say, I encountered some deja vu experiences in those projects, also known as war stories. One remarkable example always has been the problem domain itself. Even in projects with lots of experienced participants the problem domain had been often root of misunderstandings and misconceptions.  In large projects with outsourcing partners, external partners and other suppliers common knowledge of the problem domain is essential. A project  with lack of problem domain knowledge among the stakeholders is doomed to fail. Remember the tower of babel where lack of communication and insufficient common understanding lead to lethal catastrophe.

As I am mostly acting as software architecture mentor and coach or technology expert I am often master of the solution domain, but not of the problem domain. For me it is inevitable to obtain a detailed knowledge of the problem domain. This is even more important when developing product lines or platforms, because in that case lack of common understanding influences more than one product or solution.

How could we address that problem? My recommendation is to introduce a domain model. Of course, I am referring to the problem domain in this context.  A domain model is not just a glossary of terms – a common misconception I am often facing.

A domain model consists of all core concepts (actors, subsystems, components, services, …) relevant within that domain as well as the relationships and interactions between these core concepts.

 

A large domain typically consists of subdomains. For example, a telecommunications domain for Unified Communication might introduce subdomains such as basic communication or presence. As a consequence, the domain should be partitioned hierarchically into subdomains.  

There are different ways to descible a domain model, from informal approaches to strictly formal definitions such as a domain-specific language. In all cases, stakeholders might select textual or graphical representations, whatever is more appropriate.

Note that I have not assigned the responsibility of introducing a domain model to software engineers. Defining a common domain model should rather be a joint endeavor of all stakeholders, maybe driven by requirements engineers and architects.

If you have ever read definitions of software architecture like the one by ANSI/ISO, you might recognize that architecture definitions match closely with my definition of domain models. First of all, those architecture definitions are insufficient (due to their genericity). Secondly, a domain model is a good starting point for creating the architecture. Take the use cases (black box view) and specify their dynamics using the domain model entities (white box views). Supplement this architecture core with design tactics for non-functional requirements. This will lead to the architecture baseline. Using the domain model also helps you staying focused on the problem domain instead of diving to the solution concepts too early.

Eventually, a domain model defined, understood and agreed by the various stakeholders such as requirements engineers, developers, testers, architects, product managers, customers serves as a fundamental mean for architecture design and communication. 

Of course, the solution domain itself can also be expressed by a domain model. This is why the main task of an architect consists of mapping a problem domain to a solution domain.

Formalizing both to a sufficient extent helps automating this process using model-based software development approaches. Such automation, however, might be too expensive for one-off applications or smaller product lines.

Whenever you are participating in a development process in the presence or future: do not forget to specify the domain model if not already available.

Podcast on Software Architecture

For those of you capable of understanding German, I’d like to point you to our new Podcast on Software Architecture called Software ArchitekTOUR. You may also search in the ITunes Store for the podcast and find it – for example, search for ARCHITEKTOUR in the podcast category.

The Podcasters are

• Markus Völter (freelancer, associated with ITEMIS)
• Stevan Tilkov (innoQ)
• Christian Weyer (thinktecture)
• Michael Stal (Siemens)

To stay in touch with reality we have split up the podcast in three subcategories:

• General Architecture Topics (sponsored by IBM)
• Java and Software Architecture (sponsored by innQ, itemis, thinktecture and Siemens)
• Microsoft Technologies and Architecture (sponsored by Microsoft)

There are already 4 episodes available. Episode 0 is more intended to introduce ourselves. Episodes 1-3 cover patterns in general, as well as patterns in Java respectively .NET.

 

Listen, enjoy, and give us feedback

Simplicity rules

Many architectures are overwhelming. Not that they shine due to their excellent quality. Rather they appear as a big ball of mud. I already mentioned what I am calling design erosion in the context of architecture refactoring.

To keep an architecture small, expressive and simple we can do a couple of things as software architects.

For example:

• Each and every design decision must be based on an architecturally-relevant requirement. There is no decision without documented reason. Of course, this requires architects to feel responsible for the quality of these architecture requirements, but not for requirements engineering, of course! It is recommended practice to keep track on where in the architecture which requirements are “located”. Requirements traceability is an essential value, especially when assessing, refactoring or evolving a system. I am assuming here that architects already checked for the feasibility and essence of the requirements from a business perspective. Availability of 99.99999% is not reasonable in a project with a budget of 100k bucks. 
• Introduce symmetry and conceptual integrity. This includes the prescription of specific patterns, the introduction of design and coding guidelines, as well as the introduction of guidelines for (at least the most important) non-functional qualities. This prevents that the same problem is solved in various ways within the same solution space and it makes sure, we are applying best practices from experts instead of reinventing the wheel. War story: A system where designers have introduced dozens of ways for fault handling will be inexpressive and complex.
• Don’t consider technologies as toys. Many projects play with technologies. They base their decisions on a technology-first approach. This is human and we as software engineers like to experiment with the new hip operating system or SOA middleware. But in practice, we should first create an appropriate architecture meeting the requirements (see first bullet point) before we should start introducing new technologies. The business and the requirements drive the architecture, not technology.
• Test technology. The previous point does not mean we should not care about technology – only that we first address the requirements. Every technology must be checked for its appropriateness. If your task is to build a high performant messaging backbone, then you better check whether EJB is really the proper platform to use. Don’t trust any vendor’s promises here! They do not know your problem context and system environment. Building throw-away prototypes is the best way to check technologies.
• Apply a test-driven development approach: this is to introduce a kind of safety net. Emphasize testability in your architecture design, e.g., by proper modularization, interfaces, and service contracts. Test-driven in this sense also includes regular design and architecture assessment where you try to identify architecture smells.
• Regular architecture refactoring helps address architecture issues early. When a dependency cycle has just been introduced unintentionally, it is not a big issue to detect this kind of design erosion early with architecture reviews or architecture analysis tools. You are then able to get rid of the problem early by architecture refactoring. The more you wait, the more architecture entities will make it difficult to address the problem without heavy impact. Thus, do regular architecture reviews and refactorings.
• Strategic before tactical design. Don’t try to introduce over-generic solutions with myriads of configuration options. First address all strategic requirements (functional requirements and operational requirements) before dealing with tactical issues such as modifiability. Use the Open/Close principle to open your architecture for variation.
• Introduce priorities for all requirements/features. The priorities help decide which requirement to address first in architecture design. If security is more important than performance, you’ll end up in a completely different solution architecture, then wheny performance has the higher priority.
• Follow an approach of piecemeal growth. In all but the most trivial applications a big design upfront approach will be doomed to fail. Thus, assign requirements depending on their priorities to iterations and build your system incrementally.

 

These are only a handful of recommendations. Certainly, there are even more. Any tipps from your side?

Get off of my Cloud

It enters the stage as one of the great buzzwords of the last months and if we had not to deal with an economic crisis these days it would gain much more attention. Cloud Computing and its relatives such as SaaS are dominating the IT world. Ever heard about Windows Azure, Google Web Apps, Amazon E2C and S3? The idea behind this paradigm is rather old: why not provide resources transparently as services  in a hosting infrastructure? This would let IT departments and CIOs refrain from being infrastructure providers. Instead they could focus on their actual job: providing IT functionality to drive their company's business. 

By services I don't just mean applications, but all kinds of services.  An application such as a CRM app could be a service. Likewise, services could comprise a  piece of middleware such as a persistence solution or a workflow management system. And in the most extreme case, even the infrastructure itself, for example, operating systems, storage or network solutions might be deployed as services. Services where ever you look.

The idea behind cloud computing is a very promising one. Just take trusted infrastructure providers, let them host the applications (services) we need within an Internet-based cloud, and access these services from an inexpensive device. No need to buy or maintain your own data center solutions anymore. A data center on everyone's desktop if you like. Whenever you need more resources, just take them from the cloud and only pay for what you use. "Elastic computing" made real. Sounds like a good idea in an economic crisis, doesn't it? The strength and weakness of Cloud computing today is its dependence on external providers. Would you run your mission-critical applications with sensitive data on a 3rd party infrastructure that is not under your control and which supports multi-tenancy? What about services running in a remote country that you do not trust? How can a provider guarantee quality-of-service in an Internet-based infrastructure? Remember all the recent downtimes of Google Mail. What if all your company mail is running on such a network?

These problems imply a strategy where mission critical applications only run in a private cloud, while only a fraction of less critical applications may leverage public clouds. As soon as the Cloud Computing technology evolves and matures, more and more services might be moved to public clouds.  Then, private clouds could also be extended using public clouds. By the way, this extension approach is only feasible if standards foster interoperability of cloud solutions.

From 10000 feet Cloud Computing denotes sort of a business solution that introduces pay-per-use models for all types of services. Regarding technology this solution is based upon a bunch of existing paradigms such as virtualization, SOA, Multicore CPUs, NAS, or Web 2.0.

What all those people enthusiastic about Cloud computing tend to forget, is that there is no free lunch. It, indeed, represents a promising technology with a high coolness factor. However, from a software architecture perspective, it is rarely sufficient to just deploy application silos in virtual machines that are hosted somewhere else in the network. To really leverage clouds your applications need to be aware of the cloud infrastructure. You might split islands of functionality into interconnected services, and integrate application services with other platform or infrastructure services. What about issues such as collocation of services and data? In addition, engineers need to create desktop-like GUIs that communicate with backend services as promoted by Web 2.0. Last but not least, quality of service such as availability or responsiveness does not only depend on infrastructure and platform but is heavily influenced by the solution architectures.  The best infrastructure or platform does not help if the software architecture is a bunch of crap.

Cloud Computing offers fascinating new possibilities. But we shouldn't suffer from the hammer-and-nail syndrome. When applied inefficiently Cloud computing will inevitably lead to problems and disappointments. If your management or CIO department is all of a sudden promoting Cloud computing, this might be a good thing, but it also requires you as a software engineer to deal with expectation management. And finally, mind the architecture!

My Domain is my Castle

One of the most important (first) steps in architecture modeling encompasses the description of the domain model. This model introduces all entities relevant within the current domain as well as their relationships and interactions. It represents the main language all stakeholders should understand. All further activities in architecture modeling basically take the domain model and enrich it with additional infrastructure entities. 

Sounds very abstract, right? Let me give you a concrete example. Suppose, we are going to develop a Web store. What are the typical objects that appear in the problem space and are well known to all stakeholders?

For example, I'd expect entities such as:

• web store user: someone accessing the web store
• customer: someone interested to buy items
• shopping cart: used to add, remove, pay goods
• product catalog: presents all available products classified by categories and allows to search for these products using different types of query
• customer database: place where all customer information is stored
• order processing: responsible to process orders

If you think about this example further, you'll recognize there are some typical relationships between domain objects. For example: a web store user could be a customer or an administrator. A customer typically owns at most one shopping cart at the same time.

You also can easily defer some interactions between the entities. It is obvious there must be a relationship between the shopping cart and the product catalog, because the information about purchased items is read and updated from the product catalog. Of course, the shopping cart needs to interact with the order processing system after the customer has pressed the order button.

With other words: when thinking about how use cases would be mapped to sequence or interaction diagrams, the knowledge about the domain model is essential. It is the step taking you from a black box perspective to a gray box perspective.

There are different ways to express such a domain model. You could invent a graphical representation or use a textual language instead. If a domain object model gets formalized, we call this a DSL (Domain-Specific Language). In this case, engineers could even provide generators that map from the problem domain to the solution domain, i.e., that map problem domain objects and relations to solution domain objects and relations.

Using domain models offers huge advantages:

• it introduces a common vocabulary among all stakeholders which supports effective discussions and often prevents a steep learning curve for domain dummies
• it reduces the chance of missing to address important parts of a domain
• it eases architecture modeling significantly
• it helps focusing on the problem domain instead of always diving into the solution domain
• when enriched with domain patterns (analysis), it boosts productivity

It is neither necessary nor useful trying to come up with a complete domain model in the first place. In most cases, there will be an initial, maybe incomplete, domain model which engineers together with the other roles will evolve over time.

There are several examples, where the domain model is already available. Think about GUIs, compilers, and some kinds of healthcare domains. In these domains, it is useless and a complete waste of time to come up with your own domain model.

 

So, whenever you are being involved in a new software architecture, always mind the domain object model! It will be your best friend.

Software and System Architecture

Currently, I am involved in a large-scale project where Siemens Healthcare is building a couple of very innovative and exciting particle therapy systems. These systems are rather challenging as they consist of complex software, hardware  - for example, a circular particle accelerator - and all the other constituents such as the building, the CT devices. It is a typical project where software while being a major element is only contributing a small part of the system. However, within our company which yields revenues of over 70 billion euros a year, software engineering has become a major business factor. According to an estimation 60-70% of these revenues already depend on software.  Now imagine, 10% or even more of our projects would fail - what a nightmare! There are two implications: first of all, lead software architects must be well educated in software architecture and technology and show the right leadership skills. Secondly, system and software architecture must be treated as two sides of the same coin. So, what are the responsibilities  of software architects and system architects and how should they cooperate? I tried to find an appropriate definition in Wikipedia and failed. Especially, system architecture seems to be not well understood or ambiguous.

From my viewpoint an effective partitioning in software and system architecture depends on the characteristics of the project itself. If software constitutes the main part of the product or solution, then the lead software architect could and should also be system architect. Obviously, an additional IT architect  makes a lot of sense if the underlying infrastructure is sufficiently complex, but that's a different story.

In projects where hardware and software are of the same relevance or where hardware dominates, for example in most embedded systems, a system architect should be in the lead and co-operate with a hardware and a software architect. The system architect then should care a lot about testing. I have been involved in projects where I was in charge of the software system but got only rarely access to the target system. Thus, we had to use the development system where everything worked stunningly fine, but got deadlocks when finally moving to the target system - the OS of the target system had a faulty implementation of  the Sockets libraries. This implies, that the system architect should allow software and hardware development to happen almost independently, but introduce a lot of sync points where software/hardware integration is checked thoroughly. By the way, a similar problem occurs when partitioning a software system into modules which are then designed and implemented by different teams - think of outsourcing in the extreme case. If integration and system testing is neglected, you will encounter a lot of bad experiences. It is much more complex and challenging to check hardware/software integration than the integration of software modules - at least in theory :-;  One of the bad experiences you might need to cope with is that hardware requirements are often considered almost fixed, while software requirements are considered "soft". Would you ever change the main specifications of an airplane shortly before it is delivered? Definitely not! But this does not hold for software systems.  One of the consequences of this fact might be that the hardware team has already finished their part, being reluctant to change anything. As the software requirements have changed in the project, software development has fallen behind schedule. In these situations, software architects sometimes become the scapegoats of system development. A good system architect should be capable of handling these situations effectively and efficiently by better synchronizing the different teams.

A system architect should also make sure that the overall system requirements are mapped consistently and completely to software architecture and hardware related requirements. He or she should care about requirements traceability, especially for those requirements that have to be implemented in software and hardware as well.  If hardware comprised the core part of the system, software architects should have a sound understanding of the overall system, not just of the "soft" parts.

Basically, you could consider the partitioning of software and system architecture responsibilities in almost the same way like splitting responsibilities between software architects and the lead architect. The system architect is in charge of the whole system, while the software architect is in charge of a part of the system.

There is a lot more to say about this topic. But I am curious what you think and I am wondering about your experiences with software and software architecture as well as with the respective roles. Any comments highly appreciated.

News from the Engineering Frontier

I have been to the Canary Islands for a month. Since then I have been totally busy in two projects. One of them is dealing with medical particle therapy solutions, while the other one consists of setting up a certification program for software architects within Siemens. I am in charge of this program. Last week I was giving a seminar on software architecture with 12 guys from Siemens. I had a really great time and hope the same holds for the participants.

It is time to post more on this blog. Yes, I feel guilty for being so lazy. But you know, I am in love with Work-Life-Balance. To be honest, I needed a timeout from all software engineering business for a while, instead focusing on sports activities and digital photography. Now, I am really motivated to enter the software architecture bandwagon again. Last monday, we (= Markus Voelter,Stefan Tilkov, Christian Weyer, and myself) started with a new podcast series on software architecture. This will be in german with some company sponsors. I will keep you informed about the details when things mature. I am wondering whether a software architecture podcast in english would make sense.

In the medical particle therapy project we have just defined a possible partitioning between system architecture and software architecture. Seems to be a major source of misunderstandings in many projects. Same for differentiating problem and solution domain. Engineers seem to be addicted to the solution domain drug. Instead of thinking hard about the problem domain we always try entering safe ground, in particular by addressing solutions very early. The problem is: how can we define a solution when we don't understand the problem? On the other side, bubbles don't crash, as we also know. We could endlessly define UML diagrams without ever dealing with implementation. As Bertrand once said: all you need is code. Thus, we need to ensure the feasability of our solution very early. How can we satisfy both goals? Needless to say, agile development is the right answer.

Of course, there are a lot of additional issues when addressing the boundary between system and software architecture. This will be subject to upcoming posts, All feedback and comments as always highly appreciated

Views on a Cat

Suppose, you're going to design a new software system for a specific domain. One of the first steps is to introduce a domain object model which comprises the core entities in the domain as well as their relationships. That's easy, you might say. If it is that simple, why do so many  projects fail in establishing the right domain object model? And, of course, this challenge inevitably appears in all kinds of modeling activities.

But now for something completely different - as Monty Python would say. I got two nice cats at home. In order to model a cat, there are different possibilities:

• (ATOM) We could consider a cat as a mere collection of molecules.
• (GENE) Obviously, a cat can be uniquely identified using its DNA.
• (BODY) Another approach is modeling a cat as an aggregation of subsystems such as legs, joints, muscles, intestines.

Does it make sense to apply the first model (ATOM) in order to understand how a cat moves? No, because the detail level is overwhelming.

Is it useful to use the third model for understanding all chemical activities within a cat? Definitely not!

With other words, we can model the same physical concepts in different ways, each of these views strongly depending on the purpose of the model. Don't let yourself be confused. It is not as simple as having only one model for one purpose. For example, to understand the physical appearance of a cat, you could apply model ATOM as well as model BODY.

What are the implications of this observation?

• First of all, you might require different models (views) in order to define and understand a software system.
• Secondly, each model should be motivated by a concrete and useful purpose.
• Thirdly, all participants should agree on syntax and semantics of the model.
• And last but not least, all models should be documented.

Mind the word "useful" in the second bullet point. As organization drives architecture, the organization often directly maps to entities in the model. If your organization is badly structured, your architecture will reveal the same problem. Think about this in the context of your own organization.

As another consequence I'd like to emphasize that it is not sufficient just inventing a couple of sophisticated models. The set of models should also be simple, complete and consistent. All of the models should complement each other in a meaningful and appropriate way. That's the reason why we got view sets such as the UML 4+1 view.

Unfortunately, I often sit in project meetings where people strongly believe they are all sharing the same view of their domain, while in fact they are not. Endless discussions might be an indicator for this problem. Thus, it is essential to explicitly come up with a common domain model that is agreed among all participants. Basically, the design of such a model should be one of the early steps  in a development project (directly after scoping but before architecture design). Needless to say that software engineers are surrounded by models: they have specific views of the problem domain and the solution domain, use tools such as compilers or database systems that themselves are built on top of models.

If a software developer's life is about models and model transformations (e.g., the mapping from the problem domain to the solution domain) we should consider models as first class citizens and invest sufficient time for choosing the right viewpoints and establishing the right models.

Interestingly, even our view of the physical world such as the way we understand a cat is determined by models. But this is a different story.

Archidictatorship versus Develepocracy

In the software development projects where I am involved as an architect I  can typically observe different types of software architects. Let me illustrate two extremes:

• Some architects try hard not to decide anything or at least not too early. These architects typically integrate all other project participants in the decision process.  On one hand, this can be a very successful approach. If everyone agrees to important decisions, everyone remains motivated. On the other hand, the same habitude leads to the risk of decisions being postponed endlessly. Eventually, architects constantly drawing new UML diagrams or deep diving into technical details are a problem not a solution.  In most cases, wrong decisions are better than no decisions, because wrong decisions can be detected very early and all problems resolved. Think of refactoring as a tool!
• Other architects prefer a kind of tyranny. They got a lot of self esteem, believe to know everything, adore the Borgs for their "resistance is futile" strategy. This style of leadership can come very handy in critical situations where project success depends on strong leadership, for example when you require fast decisions. The downside of this dictatorship is that architects behaving this way may not get buy-in from other participants, thus leading to a high level of demotivation. And obviously, too much power in the hands of the wrong persons can lead to projects being doomed to fail miserably. A negative implication might be that those dictators want to get involved in every decision, even in unimportant ones.  As no one is able to understand all details in the case the project is large enough, you'll inevitably will experience lots of wrong decisions.

You will argue that these leadership styles are not specific to software architects, and, of course, you are absolutely right!

What I'd like to emphasize in this posting is the importance of leadership skills for software architects. Of course, there are situations where an architect should be dictator, and other situations where democracy might be the more appropriate choice.

Unfortunately, or should I say fortunately, all of us got different personalities. Your leadership style should fit with your personality, because otherwise everyone will recognize the discrepancy.

I won't cover leadership styles in too much detail because there are persons much more knowledgeable than I in this context. Just read this one or that one. A Web search will even yield more material.

What I consider really helpful is a kind of self analysis. For instance:

1. Personality: analyze your own personality, try to find your strengths and weaknesses. Ask other people how they assess you as an architect. How do you evaluate yourself acting as an architect? A very helpful tool might be to analyze previous projects: where did you succeed and why and where did you fail.
2. Leadership Styles: read about all possible leadership styles. Nothing more to add here.
3. Context: Think in your project context what leadership style should be applied in which situation. Of course, you can't act as a dictator in front of your head of R&D. Likewise, always striving for full agreement by every developer can result in endless recursion.

For a software architect, the time spent for communication might be as large as 50%. Communication means interacting with other people, informing them, guiding and mentoring them, or getting their buy-in. Without the right social skills and leadership style, a software architect is doomed to fail.