Monday, March 29, 2010

Implicit versus explicit

In a very larg telecommunications application I was supposed to review, the software architects had introduced a strict layering to shield different abstraction of functionality from each other. One of the lower layers comprised the database system, while the topmost layer consisted of the application UIs. Whenever something in the DBMS layer changed, the UIs broke. This shouldn’t happen according to the software architecture document. Thus, I asked the UI developer whether he abided by the strict layering. His answer was no, because he got the order to optimize performance which required him to access the data directly. What we see, is a typical example of implicit design decisions. When designers or developers break the architecture or use it in an unanticipated way without communicating the issue to software architects, this will inevitably lead to severe problems which are typically hard or even impossible to detect.

In another project the development organization built a GUI for a monitoring framework. After the application had been relaesed, customers started to complain about a feature they were expecting but which was missing. This illustrates what happens if requirements engineering or customers have implicit expectations. Unfortunately, these requirements change over the years. While 20 years ago almost every developer was satisfied with command line compilers and had to explicitly specify she would prefer more IDE-like environments, today this feature is considered an implicit requirement. You don’t need to mention it explicitely anymore. KANO-Analysis is a perfect tool for such investigations. For programming environments this is very obvious bĂșt what if you’re working in a domain where you don’t know about all these hidden requirements. Thus, mind all invisiple traps.

There are two conclusions we can draw from these examples:

  1. In a software development project, any information should be made explicit, even facts we normally believe every stakeholder should know. If communication among stakeholders is the most important asset in such projects, nothing shall remain implicit. Implicit information is unavailable information!
  2. In every project we need enforcement. Requirements engineering must enforce that the development team really knows and uses all requirements. In a specification all implicit requirements must be turned into explicit requirements, because implicit requirements are typically swallowed by black holes. Software architects must enforce the architecture by closely cooperating with developers and must ensure quality by closely cooperating with testers.

Work based on implicit information can only possibly work in small teams that use an agile approach and where every team member has an explicit knowledge of implicit information. But do you dare to take the risk that all information gaps are really closed?

Always be an explicit software architect without a hidden agenda!

Friday, March 12, 2010

Over the Fence

It is surprising how many software architects still create their design, throw it over the fence, and then expect others to follow the divine strategies implicitely hidden behind the lines and diagrams.

This behavior also holds for different teams with key developers or subsystem architects that are ought to coordinate design activities, while in practice they are happily inhabiting their isolated islands.

I’ll never forget a telecommunications project where the architects had introduced excellent guidelines and a decent strategic design. However, I was called for an architecture review, because the implementation had turned out to be not quite decent, to say the least. When I searched for the root of all problems, it became clear very soon that no one in the project had really cared about any of the architecture guidelines. And even worse, the strictly layered architecture design had been seriously violated. For example, whenever the database schemas were evolved, the clients broke, although they were totally isolated from each other by a “firewall” of 3 layers. The key developer of the GUI told me in an interview, that he considered performance optimization as his primary directive so that it semed to be totally acceptable to directly access the database.

We can learn from these war stories that good communication is the (most) critical aspect in any project. Without good architecture enforcement, there is no good architecture. But how can architects  pragmatically ensure high quality as well as conformance with the architecture design? Management by walking around does the trick. You should frequently visit the different teams and make sure, they really understand the infinite wisdom and essence in your architecture design and architecture guidelines. This is the best way to get their buy-in and support. In addition, you ‘ll detect any architectural shift early. In particular, when developers add accidental complexity by introducing design pearls that are not motivated by requirements. I wholeheartedly believe, agility implicitely requires architects to work this way. But it is not the case that architects are inerrable, (although we’d really like to see the universe from this perspective). Sometimes, even clever wizards fail. As an architect I am definitely better off when developers show me where I was plain wrong. Believe me, this is much better than testers, reviewers or customers happily finger pointing to your design flaws. Thus, architecture enforcement is a bidirectional interaction between architects and implementation, with architects being the drivers.

As Lenin once said: trust is good but control is better. Think about this before throwing your design over the fence in the next project. Some things tend to backfire heavily.

Sunday, February 21, 2010

The Extensibility Syndrome

In one of the Facebook groups I am member of there is currently a discussion going on how to deal with extensible design, especially how to prove to a client that a system meets its extensibility expectations.

Focus on extensibility is important but can also be a big hazard, in particular, when engineers overdo it. For example, consider the application of the Strategy pattern. Basically, applying the Strategy patterns implies some kind of uncertainty which algorithm actually should be used. If many Strategy pattern instances are used, this almost always means that the engineers have no clue in which direction the system should be extensible. Of course, the same holds for overuse of other extensibility patterns as well. Such systems become a nightmare for developers and users.

Thus, make clear with requirements engineering or customers what kind of extensibility they really need.

What needs to be extended:

  • Should an algorithm be exchanged?
  • Is it a whole subsystem or layer that is subject to change or extensibility?
  • Is it a component that needs to be changed or extended?

When will it be extended?

  • Now,
  • Mandatory in the future,
  • Optionally in the future?

Which binding time?

  • source code,
  • compile time,
  • link time,
  • runtime,
  • maintenance time?

Who will be in charge of extending?

  • operator,
  • user,
  • developer

You should only consider those extensions that are mandatory, none of the possible or optional ones that may or may not appear in the future.

In the beginning of software design introduce change scenarios as proposed by the book Software Architecture in Practice, 2nd edition (Bass, Clements, Kazman). Here you’ll learn how to describe modifiability scenarios using scenario diagrams and rating them using so-called utility trees.

You may also use design tactics diagrams that deal with how to implement the different kinds of extensibility mentioned above.

If you follow the principles of my Onion Model, then each architecture design step will only be driven by requirements and their priorities. This way, you can establish requirements traceability within your architectural design.

For extensibility and change scenarios, a Commonality/Variability analysis can be leveraged, especially when dealing with a wide variety of extensions such as in platform or software product line development.

In contrast to common believe extensibility or change is only applicable after you’ve designed those parts of your architecture that are subject to extension or change. You can either extend some functional entities or operational qualities such as scalability mechanisms. This implies that all extensibility and change design activities follow after functional and operational design and not before.

When opening your functional & operational design for extensions or change always use the Open/Close principle to prevent unwanted side effects.

To prove that a software architecture is extensible such as expected by the customer you got different means.

  • An end-user or operator might like to see the implemented change scenarios and how they actually work in the implementation.
  • Another engineer might like to see how the architecture and implementation support extensions.
  • A requirements engineer may like to see the mapping from requirements to architecture decisions.

Modifiability aspects (e.g., extensibility and changeability) are much more difficult than they seem at first place. When the amount of extensibility is overwhelming, you’ll often find wizards or configuration automatisms to hide the mess underneath (which does not imply that wizards or configurators are indicators for bad design).

As Heraklitus once said: panta rhei – everything changes. Thus, be prepared!

Friday, February 12, 2010

Does Cloud Computing have an Architectural Impact?

Let’s suppose you have good reasons to leverage a Cloud infrastructure. Will this decision hava any impact on your architecture? The answer is quite easy: It depends :-)

  • If you just use virtualization to exactly simulate a physical environment like in Amazon EC2, then your applications will be oblivious to the Cloud infrastructure.
  • If you leverage SaaS (Software as a Service) like Salesforce.com then it’s a SEP (Somebody Else’s Problem).

However, if you use PaaS (Platform as a Service) or IaaS (Infrastructure as a Service) for your applications, then things definitely will change.

It is not so much the domain-specific logic that is influenced, but infrastructural and non-functional requirements will experience an impact.

  • In a Cloud environment different storage means are supported. In general, you’ll find Blobs and NoSQL databases. Designing the persistence infrastructure must take these issues into account unless you are still relying on a mainstream DBMS running somewhere in the Cloud.
  • For messaging Message Queues are available. Thus, message-oriented communication between Cloud-aware subsystems is a reasonable approach for connecting islands with each other. Often, SOA style communication protocols such as RESTful communication or SOAP/WS-* are supported. These protocols alone imply different architecture paradigms - designing the communication and distribution infrastructure can be a challenge.
  • All operational qualities such as performance, availability, scalability, fault-tolerance need to be implemented with the SLAs in mind the Cloud provider offers. If, for example, availability zones are supported in the Cloud you better plan how to leverage them according to your needs. In a public cloud security is even a more critical issue. You cannot naively trust the Cloud provider to keep your mission-critical data secret. Thus, you have to provide means like encryption almost everywhere. For performance reasons, you might want to consider the Cloud as a Grid such as in Apache HADOOP. But how can you partition your system in a suitable way. Is Mapreduce what you need or, otherwise, does your application require a different kind of approach? There are many different concurrency architectures like Master/Slave, Pipes & Filters, Concurrent Reactor that could be appropriate. In contrast to typical application development you are much more constrained when addressing these operational qualities, because many decisions the Clound infrastructure has already made for you. It is YOU who needs to integrate with the Cloud, not vice-versa.
  • Developmental qualities such as modularity, modifiability, maintainability can be hard to achieve in such an environment. First of all, you should introduce some kind of governance approach. Secondly, you need to modularize in such a way that governance issues live in harmony with the kind of modularization propagated by the Cloud, in particular by the PaaS APIs. For command & control purposes you need to think about how to administer the Cloud applications. There might be 3rd party products for this purpose. It, however, might also be the case you have to implement your own tooling.
  • If, for some reason, you need to use different cloud infrastructures, interoperability might be a big issue given that today’s approaches almost inevitably lead to vendor-lock-in.

Not always, you will be able to start with a green field project, but rather need to deploy an existing application ecosystem or parts of it to the Cloud infrastructure. Migration is not straightforward as we already saw earlier. You may need to combine reengineering and refactoring activities for this purpose.

This posting describes only the tip of the iceberg. I hope, people will develop some best practices (patterns) for how to leverage existing Cloud platforms. I’d be rather sceptical if someone would come up with such patterns soon. You surely remember, patterns must be independently used in three different applications to make them patterns. I cannot imagine, we already obtained that much experience in practice.

So, if you are going to design for Amazon EC2, Google Web Engine or Microsoft Azure don’t forget: Mind the Cloud!

Thursday, February 11, 2010

Requirements traceability – The Holy Grail

It often occurs that I am supposed to review an architecture document – that’s one important part of my job. After having read the document, I always try to summarize what I just read. And in many cases, this proves to be rather difficult. One of the reasons is that the subsequent parts of the document seem to be only loosely connected such as graphical models popping up without textual explanation or architectural & technical decisions appearing without any hint what the design rationale could be. However, a reader who is dependent on rumors, insider knowledge, and vague assumptions will see the hodgepodge of syntax but never fully understand the semantics behind this syntax. If the reader is a tester, requirements engineer, fellow architect, or developer such situations need to be considered harmful. Yes I can see the answer is 42, but was the question respectively problem that lead to this result?

The big secret is requirements traceability. Each and every architectural decision must be strongly derived from forces (i.e., requirements and risks). This also helps to keep the architecture simple, expressive and balanced, because this way we can get rid of all these design pearls smart developers and architects typically invent.

From the forces elicitated at the beginning of a project architects derive architecturally relevant requirements. This includes requirements clarification and priorization. Remember: Garbage-in-Garbage-out. If the initial assumptions are wrong, the resulting implementation can under no circumstances meet the expectations.  Of course, we do not expect requirements to be fully available at project startup. We have become extraordinary agile wizards, anyway, and can also handle requirements dropping in. Needless to say, we can not gurantee the same perfect software & system architecture we would have come up with if all these requirements had been fully available from day 1.

As we already got those long-term experience, we are using the onion model.

  • For all use cases (part of user requirements) we are deriving a black box view of our system. From this descriptions we also obtain actors and required/provided interfaces. As we see, this step is driven by requirements.
  • The domain model is another force. It is a means to understand typical entities in the underlying domain as well as their relationships and interactions. They will turn the black-box view into a white box view. Now you can start to map the use cases to sequence diagrams which is also requirements driven. All of a sudden the domain-specific subsystems, components, interactions, interfaces are disclosed refining our external model. Eric Evans is an excellent source how to deal with such domain-driven design.
  • So far we have focused on the domain entities. It is time to add infrastructural requirements such as communication, persistence and concurrency infrastructures as well as operational requirements such as performance, safety, availability. This we do by starting with the highest priorities- that’s why priorization is so essential – and ending with the lowest priorities. Each architecture decision is strongly rooted in requirements.
  • The same is done with the mandatory developmental requirements like modifiability, maintainability and then with the optional developmental requirements.

All these decisions are depicted in diagrams, but also explained in the textual descriptions.

What are the benefits of such an approach:

  • We get requirements traceability: Firstly, we know which requirements lead to which architectural decisions, thus also obtaining a mechanism to check whether we have already covered all the necessary requirements in our design. Secondly, we know which effects an architectural change such as a refactoring will have on requirements, thus preventing or at least lowering the risk of accidental complexity. Thirdly, all decisions are made explicit. Implicit parts lurk like invisble ghosts deep in the guts of the architecture hurting us when we are least prepared.
  • We are driven by priorities. If we miss our milestones, we skip the less important requirements instead of the really important things.
  • Readers of our documents will not only understand the “what” but also the “why” making it much easier to get their buy-in. This is what we call design rationale.
  • If we additionally come up with leading guidelines and principles for cross-cutting concerns in the beginning we will increase internal qualities such as symmetry, orthogonality, simplicity.

The journey is the reward in software architecture. Don’t tell readers about the target to aim for, but also about the way how you get there. On each junction, explain why you prefer one over the other road.

Writing of such documents can be such a fun. And reading them even more. For example, read Dave Cutler’s book about the creation of Windows NT.

A architecture document shall contain guidelines and design rationale, but no labyrinths.

Saturday, January 30, 2010

Back from the OOP 2010

This year’s OOP conference in Munich has been a lot of fun. I met Markus Völter, Stefan Tilkov, Floyd Marinescu, Jan Bosch, Eric Evans, Philippe Kruchten, Roman Pichler, Peter Hruschka, Kersten Auel, Rene Schönfeldt,  Klaus Rohe, Matthias Bohlen, Gernot Starke, Jutta Eckstein, Nico Josuttis, Gregor Hohpe, Robert C. Martin, Matthias Bohlen, Kevlin Henney and my good ole friend Frank Buschmann. Especially the networking part was overwhelming. So many speakers and participants you can learn from.The talks and keynotes are definitely helpful to hear about new trends, perspectives and experiences. But the personal communication is the essence of such conferences. That’s the reason why I don’t believe in online events. Sure, it’s a nice add-on but it cannot substitute the “human” factor. In the middle of the week we had a meeting of the Siemens Senior Software Architects where I enjoyed meeting some of the aforementioned celebrities but also the smart senior architects within Siemens who do excellent work day by day. They might not be that much in the limelight, but those people are  the real heroes of software engineering among all those “nameless” other heroes such as development heads, team leads, testers or developers. As Jan mentioned: there are two types of architects. Those talking about software architecture and those practicing it. Fortunately, I am a hybrid belonging to both types.

This year’s keynotes  offered excellent quality. I’ll never forget Uncle Bob’s entertaining talk on the polyglot programmer where he dived into the history of programming languages in a very entertaining way. Or two speakers from ZĂŒhlke Engineering who presented a great keynote on functional programming. Not to forget Gernot Starke with his talk on software architects and stealing in your neighbor’s garden. Unfortunately, I could not make it to his keynote but from what I heard it was one of the highlights.

Most of the topics presented in talks and tutorials covered the typical topics you’d expect. Cloud Computing and Functional Programming as well as Agility represented the hot topics. What I liked most were all these more practice-oriented talks offering some insights for practioners and also revealing some real life show cases. For instance, I enjoyed Kurt Höpli’s talk on building and controlling environmental sensors in a project in Switzerland. And I also experienced a lot of fun in Markus Völter’s talk on how to apply Model-Driven-Software-Development  within an embedded systems context. Never I will forget how Eric Evans and Hans Dockter presented Domain-Driven-Design convincing some attendees to participate as actors. And I’ll remember also Lothar Mieske who presented the real look-and-feel of all those Cloud Computing platforms. I could add much more but this should give you an impression and suffice as a pars pro toto.

Personally, I have been very busy the week. First, I gave a full-day tutorial on Software Architecture – From Requirements to Architecture. It was the tutorial with the largest numbers of attendees. This makes interaction somewhat difficult but nonetheless worked much better than expected. My talk on Hitchhiker’s Guide to Cloud Computing had about 80-100 attendees. To my pleasure the Global Technology Field Cluster Lead of my organization also attended the talk. For me, it was the biggest surprise that the talk “Introduction to Scala” had attracted so many people. I was totally electrified and I mean this literally cos’ static charge hit me whenever I pressed the page-down key. When people are interested, I will record this talk with Camtasia and publish the video clip in YouTube and other media. All in all, I got really overwhelming feedback. Thank you to all attendees for their nice evaluation sheets :-)

Next year the OOP will celebrate its 20th anniversary. Hope that many of you will join the event. It is really an extraordinary experience.

Thursday, January 21, 2010

Stakeholder-specific Communication

Communication is one of the most importants aspects of software architecture design. The architecture itself is a means of communicating design decisions to customers, product managers, project managers, testers, integrators, and developers. However, these stakeholders have different expectations regarding the software architecture. While developers need a clear guideline how to implement the architecture design, customers may be more interested how (well) the software meets their requirements. The software architecture documentation and all other means of communication such as presentations must cover these various perspectives. To address such broad spectrum of interests, architects could provide a specific document for each particular kind of stakeholder which leads to an unacceptable amount of effort and time. This is why documents should rather explain the architecture decisions - the "what", the "how" and the "why" - in a top-down approach and add a guide-to-the-reader motivating who (i.e. which stakeholder) should read what in which order. For presentations or other forms of communication it might be worthwhile to introduce a specific variant for each stakeholder. A project manager or customer is typically not that interested in all those strategic and tactical design destails, while developers need all the information they can get. Thus, communication between architects and other stakeholders should take the stakeholder-specific interests into account. This implies, that architects need to have a clear understanding of these interests before starting documenting or communicating. Otherwise, they won't get buy-in for their software architecture by these different stakeholders. Personally, I have experienced so many projects where managers couldn't follow and understand what the architects told them overwhelmed by details they did not (have to) care about. And I also have been in so many meetings where software architects introduced high-level abstractions that could not provide the details developers and testers expected.
As a prerequisite it is necessary to ensure a common understanding of terminology. Did you ever attend a meeting where different participants had a different understanding of terms? Communication will not work and succeed when one person is speaking english, while the communication's target can only understand spanish. This might be obvious for real languages, but it is often underestimated for domain-specific languages. Did you once try to read and understand (!) a treatment record of your physician? He speaks english, you speak english, but this won't help you much. As a consequence, introducing a domain-specific language is valuable and essential. It does not have to be a full-blown language in the strict formal sense, but often it makes sense to offer more than just a glossary. The core functional architecture should represent the problem domain.
As a consequence, all communication between architects and stakeholders must happen in a stakeholder-specific way. When giving a presentation on the software architecture, make sure you know the target audience and their interests. Don't assume, other types of stakeholders understand software architecture the way you do. You are the doctor and they are the patients, so-to-speak. Communication may succeed or fail. The risk of failure can only be minimized if you can take and understand the perspective of your communication partners.
Obviously, these statements also hold for other kinds of communication. Have you ever tried to explain to your spouse why you need that cool new gadget for several hundred bucks? Try that by telling her/him the technical specifications. Won't work very well, right? What we take for granted in such situations (i.e., adapting the communication to your communication partner), represents also a conditio-sine-qua-non in software engineering. So, better mind the stakeholder-specific communication in the future :-)

Saturday, January 09, 2010

Real Software Engineers

  • Real software engineers don’t need safety nets. Instead, they are ready to face all dangers and risks without any fear. For this reason, they dislike (unit) testing, because (unit) testing is for pussies (i.e., those who do not trust their own code.) Since real software engineers never make mistakes, testing would be a mere waste of time.
  • Real software engineers won’t document their design, because these designs contain an incredible amount of design pearls other mortals would not be able to comprehend, anyway.
  • Nor do real software engineers read other design documents nor manuals, either because their fellow engineers did not provide documentation or because they do not trust these documents.
  • Instead, their main guidelines are:
    • All you need is code
    • Bubbles don’t crash
  • Real software engineers do not check or prioritize requirements. Instead, they got intuition for customers’ needs which drives all their activities.
  • Real Software Engineers do not communicate (much) with other stakeholders, because other stakeholders are just obstacles on the way to project completion. That’s also the reason why real software engineers hate meetings.
  • Real software engineers do rarely speak English, German, …. They rather express themselves using C#, Java, C++, …
  • Real Software Engineers feel more comfortable in the solution domain. Their goal is to provide solutions instead of getting lost in problem domains. This is why they prefer playing around with solution technologies.
  • Real software engineers can turn any problem into a nail for which they can use their favourite hammer.
  • Real software engineers do not need software architecture because a predefined architecture would just limit their creativity. A good architecture is always created ad-hoc for solving real problems in a pragmatic way.
  • Real software engineers do not foster re-use. First of all, re-use has never worked in practice. And secondly, re-using design or code typically requires more resources than reinventing the wheel. This is often due to the fact that re-usable artifacte were created by other real software engineers who failed to document their work.
  • Real software engineers won’t read this blog because in the time needed for reading, they could add another feature to their software.

Thursday, January 07, 2010

Architecture Documentation Revisited

Did you ever read a novel which later has been used as foundation for a movie? I bet, you were rather disappointed how the movie differed from your reading experience and thus from your expectations.

Did you ever read an architecture document – yes, there is vague evidence and hope that something such as architecture documents exists – and then compared the documented design with the implemented system?

In software architecture design, documents are not just by-products created for write-only brain dumps.  Instead, they are supposed to be read by someone – a fact that might be surprising for some engineers.

Architecture documents serve as the basic means for communicating architectural decisions, i.e. they explain the “what” and the “why” – aka design rationale. Target audience of architecture documents are foremost all stakeholders, not just the authors themselves.

In addition, those documents should be up-to-date. Did you ever struggled through 100s of pages of a design document, and right after you finished, someone told you, the document was outdated?

Does this mean, we should adapt the documentation after each change? No, it only implies, architecture documents should be updated regularly. Consider versioning architecture documents in this context!

An architecture specification should also be subject for testing. Don’t take testing too literally! What I suggest is that documents are being reviewed for their usability/readability, internal consistency and completeness, as well as their consistency with the actual implementation. Needless to mention, that the reviewers shouldn’t be the authors but a selection of stakeholders.

Always use the right style of documentation depending on purpose and target audience. A user document that describes a system should contain tutorials as well as a reference manual. If no one understands it, the best design ever won’t help you much.

My strategy for documenting is to walk to the other side of the fence thinking about how a specific stakeholder would like to get the material presented. I am not writing the document for myself but for others, anyway. And I always take care of updating the documentation whenever the system has been updated to a new (minor or major) version. Readers should not be punished by reading my document but consider this as an entertaining and profitable activity. Reading design documents is fun or at least should be! Who claims, IT documents must always be written in a technical and boring style?

Remember: the most important capability of a software architect are communication skills. Documenting is one particular way of communicating.

Wednesday, December 09, 2009

A picture can say more than thousand words - a word can say more that thousand pictures

One of the key problems in architecture documents is the idea illustrations are sufficient for explaining design.

Let me give you an example:

Eo---oP

What could the diagram illustrate?

Could this describe a hydrogen atom consisting of a proton and an electron?

Does it represent the association between an entity bean and its persistence store?

Or is it something completely different?

Diagrams are models. They can only express a subset of reality. If you do not know the concrete context, you’ll have no way to figure out what the diagram actually means.  But if you are familiar with s special context, you might be inclined to map a diagram to your own context which, however, might be painfully wrong.

Thus, an architect should always explain the context of a diagram as well as the constituents and their meaning textually in the document.

What about the other way round?

Again, an example:

Component A is connected with B, C, D. D is connected with E and C. F is connected with D and A … a lot of more information …

In the example above, plain text is not really helpful. Here seems to be a good place to add a diagram that graphically illustrates the components and their connections. Text alone is not capable of describing complex structures which often appear in architecture design.

Hence, we recognize that diagrams need explanations to understand their semantics. Otherwise they just represent syntactic glibberish. In addition, UML, SysML and other graphical notations might lead to ambiguous illustrations which are open to interpretation. Thus, tell the reader what exactly the model behind the diagram is supposed to be.

As a consequence, all design documents that are automatically generated have almost no value.  For example, design rationale (such as requirements and their association with architectural decisions) is better described using textual language instead of diagrams.

Thus, use diagrams carefully in your design documents. But also be aware of text that does only describe what is depicted in diagrams, but does not introduce information such as pre- and postconditions, invariants, contextual information, design rationale, …

Guess, why there are UML use case diagrams that illustrate a set of use cases, but no graphical means for describing the use cases themself?

Use text and diagrams wisely! Don’t consider design documents as write-only-entities. You will see what I mean whenever you have to understand  documents of your fellow engineers.

Sunday, November 29, 2009

Maturity matters

This time a rather small posting only.

I am wondering what the top research fields in software architecture currently are? Of course, fields such as MDSDL/DSLs come immediately to my mind. Cool stuff! Investigate these topics and fame is guaranteed!

Others are already sufficiently mature. I am thinking of mapping requirements to architecture decisions. Ok, maybe there are still some deficiencies in this area. Hmmh, there might be even significant gaps, especially when dealing with operational and developmental qualities. Must reevaluate my opinion.

What about documenting software architecture? There are dozens of templates available, even books. A lot of agreement here, but what about modeling? Is UML the “ultimate ratio”? How can we model, what should we model? How can we cover design rationale? How deep should we go? Did I ever read a really good architecture documentation?

I got it. Patterns are super mature. I have been author of pattern books myself. These books are now available since 14 years. Some architects even know the names of all patterns. But do most architects know patterns others than GoF? And even if they do, do they apply them?

Isn’t refactoring a good candidate or reverse engineering? Architecture refactoring isn’t established in most development organizations. If I say architecture refactoring, I really mean architecture refactoring, not code refactoring.

What about software architecture design? We have been designing software systems for decades and they work! No, most of them suck! Unfortunately, design by accident is the norm, not the exception.

… thinking, diving deeper, diving broader, getting frustrated …

Why is my impression that most areas in software architecture are still rather immature?

Maybe, the answer is that software architecture itself is a big collection of research topics.

Or maybe, I am blinded by the light.

Tuesday, November 10, 2009

Architect – what architect?

In many projects and organizations I am constantly meeting software architects. Given the quantity I’d only expect only a fraction of software development projects to encounter any issues in software architecture and design. As we all certainly know, quite the opposite is true.

Ok, nice and true words, but what exactly is the reason for this observation?  

Of course, I absolutely have no clue :-; So let me just make some assumptions.

Could it be that the term “software architect” on a business card is subject to a variety of different interpretations?

In one company, someone may be promoted to software architect, just because she/he is the only one who owns a pattern book. [Beware of humour!]

Consulting companies may entitle some of their engineers as software architects, because they can charge more for architects than for developers.

A handful of companies may offer a challenging education program to become a software architect.

With other words, it is not clearly defined what a software architect is or should be. Nothing prevents you from calling yourself a software architect. You may remember that I am proposing a rather tough profile for software architects. Well, I know too many projects that failed  miserably because of lack of competencies, experience or skills by the responsible software architects.

Note: not only the skills count! As software engineering is heavily related to communication and learning from failure, becoming a software architect requires starting as a developer, taking over more and more responsibilities over the years, before finally turning into a butterfly.

Another issue I often encounter is the missing separation between software and system architects. Today, many system architects are also considered as software architects. This might work very well if the system architect is an experienced software architect. Likewise, the aforementioned setting is adequate for software-intensive developments. But here again, the system architect needs also to be an experienced software architect. If the latter doesn’t hold, the software relevance is obviously underestimated and will inevitably lead to trouble.

Hold on, there is even another possibility. Even if you can justify that you are a software architect now, this doesn’t mean you’ll be a software architect forever. One of the most fundamental challenges is to know the state of the art of relevant software technologies and methods as well as appreciating and practicing coding and testing skills. Guess, you have been appointed as a software architect ten years ago, but in the meantime you were constrained to using UML tools, Powerpoint and other Office applications. With other words: would you trust a surgeon to operate on your heart who lacks ten years of practice?

I bet, you made some similar experiences in your environment. If that is the case, a comment would be highly appreciated. If not, even more! 

Friday, November 06, 2009

Sick Architecture

You may wonder why my publication rate in this blog has decreased dramatically. I am staying at home due to an infection for three weeks now which wasn’t that helpful for my blog.

To keep things running I have decided to create a new posting. Well, the title is intentionally related to my current health situation.

To be honest, I think that software architecture design reveals some interesting commonalities with sickness:

  • Both often require intense treatment by experts
  • Different experts offer different opinions and treatment plans
  • Both are typically open-ended to some extent
  • Both are not subject to planning in most situations
  • Both mostly feel very miserable
  • To reduce the risk of getting infected some kind of prevention strategy is always a good advise
  • It is better to cure the cause not the symptoms. Thus, be aware of all interdependencies and follow a holistic approach
  • Viruses are a problem in both worlds
  • Healthcare never ends – it is important in the whole lifecyle
  • Learning from failure is recommended to avoid similar problems in the future
  • Prevention is always better  (i.e, cheaper) than healing

Most software architectures I know are not in a good and healthy shape - at least in some intermediate phases. Why? Because they were not devleoped with the aforementioned bullet list in mind.  Thus, health monitoring (i.e, reviews) and treatments (i.e., refactoring) should be an essential tool in the architect’s framework.

Does this sound reasonable?

Thursday, October 15, 2009

Data centric Architecture

In most projects there is a big emphasize on functionality. Use case modelling defines responsibilities of the system as well as how it interacts with actors. Logical modelling then maps the use cases to related subsystems/components which, of course, is functionality based. At the end we'll get functional interfaces between subsystems or between components. Do you recognize that methods such as UML mostly focus on methods?



However, in many cases the data exchanged between architectural entities is rather complex. For example, in healthcare scenarios complex structures with patient information or charging information are the rule. Not dealing with data as first class citizens might lead to various issues such as versioning problems, or inadequate partitioning or aggregation of data. Thus, architects should deal with data modelling very soon if they are facing complex structures in their development projects.



Data modelling starts with the domain model. What are the main entities in your problem domain and which kind of data do they need to persist, share, or transfer?

Then in the next "layer" when we are introducing the subsystems we'll have to think about data structures in subsystems and data structures exchanged between subsystems. For example, what exactly are the data structures required in the functional interfaces. Obviously, these data structures can be partially derived from the domain model with further data structures required by infrastructural entities or non-functional qualities.

In the component layer we are starting to deal with finer grained data types.

Last but not least we'll have to define how data is mapped to implementation artifacts.



Thus, we should also follow a top-down approach for data modelling. And likewise we need to consider bottom-up constraints such as special data requirements of infrastructures or platforms. With other words, data modelling must go hand in hand with all the other architectural modelling activities:



Important questions to ask when modelling data:


  • Who is the originator of data?

  • Who is the consumer of data?

  • How does data move from the originator to the consumers?

  • Do we need to collect/aggregate data from different sources (transfer objects)?

  • What about quality constraints such as performance (e.g., caching data), security (e.g., encrypting data), modifiability (e.g., versioning data), concurrency (e.g., locking data), availability and reliability (e.g., replicating data, data persistence)?

  • Where and how is data stored?

  • How do different data structures depend/relate?

I think, data modelling is an underestimated and undervalued activity in architecture design. What about your experiences and opinions?

Friday, September 18, 2009

Ineffective Communication

It might not be that obvious why time management is of high importance especially for software architects. But of course, being a software architect in a mission critical project requires your full attention. However, in a couple of projects I met software architects who were simply overwhelmed by communication requests and finally got stuck in a kind of waste time singularity. Ineffective communication can significantly reduce your productivity. 

Unfortunately, there are many opportunities where your focus on the real important things is easily lost.

  • My favourite are unnecessary meetings (including telephone conferences).  As software architects are caught in the gravitational center of the project they receive a lot of meeting requests, by managers, developers, other engineers. If you are a very communicative and friendly person – and I bet you are – you might be inclined to say “yes, we can” to all those nice invitations. It is often overseen that meetings also require preprocessing and postprocessing activities. Hence, don’t just count the time you’re spending in the meeting room! Consider carefully what type of meeting you are asked to attend. Is it an essential meeting where your participation is absolutely required (class A meeting)? Or more one of those nice-to-have-meetings? Only accept class A meeting invitations! Even for class A meetings ask yourself whether this meeting has the right setting in terms of people invited, time frame, goals, agenda. With other words: is the benefit you obtain larger than the time spent? Sometimes, a small face-to-face meeting offers more value than a large official meeting. And of course, act thoughtfully if you invite to meetings yourself! As an Outlook junkie do not only enter meetings in your calendar but also time slots for your actual work. Mind the fact that other people tend to fill the gaps in your calendar assuming those gaps are indications of idle time. Personally, I also enter slots for my spare time, just in case. Needless to say that you should also consider travel time in this context. Care about meeting netiquette: there is no place for cell phones or notebooks in a meeting – a few exceptions granted.
  • Another critical challenge is e-mail. Communication by e-mail can be very ineffective (but also the most effective means in other situations). Try to reduce corporate spam which consists of all those mails which steal your time for no good reason. Organize your inbox in folders and prioritize mails as soon as it arrives. Process the mail depending on its priority. Read mail only at specific points of time and tell your communication peers about this fact. And do not send unnecessary mail yourself! Before writing an e-mail, always hold on and consider whether you would also have sent an letter containing the same information in the stone age of communication.

At the end, it is the project success that counts. Good results require effective architects with as much productive working time as possible. Productive hours depend on effective communication. It is the quality not the quantity!

Saturday, September 05, 2009

Vacation - the end is near

The last weeks I have relaxed a lot: visiting Dresden, running, swimming and biking, as well as practicing with my DSLR. In addition, I read two books on Scala, updated an article on D (the language), installed several apps on my IPhone (yag – yet another gadget),  prepared slides for ADC 2009 and OOP 2010, participated in a new episode for our SoftwareArchitecTOUR podcast, … and spent a little time on further expanding my work on architecture refactoring.

However, I intentionally did not work on software architecture or on this blog. This will change beginning with upcoming monday. Now, that I got some new and fresh ideas, I can continue my journey on software architecture issues.

By the way: I did not mention it so far but maybe you like to follow me on twitter (username: stal_m).

Friday, August 21, 2009

And now for something completely different

This is my first week of a 3 weeks vacation. It is really hot in Munich why I am currently enjoying swimming, biking and visiting beer gardens more than software engineering.

Nonetheless, I am currently educating myself in a new programming language. Scala by Martin Odersky combines a pure object-oriented langauge with functional features. And it runs on the JVM. An earlier version of Scala does also support the .NET CLR.

What I like most is the proper integration of functional features. Closures, for comprehensions, nested functions, pattern matching are notable examples. While Scala is statically typed (it is not a dynamic language), it offers type inference for variables, arguments and results.

In addition, it provides some cool extensions to Java: all types are objects, no static members, support of covariance and contravariance, to name a few.  Traits are available instead of interfaces – they can behave like interfaces but also allow implementations of methods. With traits the problem of multiple inheritance (mind the diamond configuration) can be avoided. This is possible by introducing inheritance linearization and a kind of virtual super.

With all of these features Scala code can be much more compact than Java - and it interoperates well with Java components.

Well, I am really kinda enthusiastic. If you ask me, Scala could be a strong competitor for Java and C#.  But of course, there is not a big company backing Scala such as Sun or Microsoft. Thus, the community will decide.

Loving the JVM but preferring other languages than Java? No more excuses, Dave!

Saturday, August 01, 2009

Neverending Story

One of the prominent questions I often hear in software development projects is when to start with architecture design. Sometimes, architects hesitate to start until almost all requirements are available in high quality. This strategy is highly recommendable when you like to postpone architecture design forever. There is not a single project with medium to high complexity I remember where we knew all forces in full depth and breadth at project start.

First of all, it heavily depends on the software development process. In a Waterfall model without iterations you are stuck anyway. Thus, I generally recommend an iterative-incremental approach. In software developments with all their volatility only piecemeal growth can succeed.

It is important to have a sufficient quality and quantity of requirements available, for example, the most important 30%-50% of use cases and non functional qualities with unique priorities. This way, you can come up with a conceptual architecture which you can then use as a base for iterative-incremental refeinement.

After the baseline is available prioritize all user requirements (use cases), map them to end-to-end scenarios and integrate them into the architecture, keeping the non-functional qualities in mind. To deal with non functional qualities let key developers, architects or even teams work on principles and guidelines how to address these qualities. Do this after the baseline architecture design but before starting the end-to-end iterations.

As soon as new requirements appear or have been concretized assign them priorities and put them into the “backlog”.

To introduce a kind of safety net for embracing change, introduce test-driven design methods, flash architecture reviews after each iteration, and (architecture and code) refactoring activities after each flash review.

Keep in mind that sometimes you need courage to decide even when not all details are available.  It is typical for all creative and volative acts that not all details are known in advance, nor the ways how to cope with them.

As a consequence, it is much more productive and effective to follow wrong paths sometimes instead of following no path at all.

To keep it simple: Prefer acting over reacting

Thursday, July 23, 2009

May the force be with you

Sometimes it is getting confusing. There are so many influential factors driving software architecture. Here is my small domain language
  • Forces is the set of all influential factors an architect is facing: requirements, risks, constraints, business needs.
  • Requirements are the (documented) set of all expectations for a software system. According to Wikipedia a requirement represents a necessary attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user.
  • System requirements address the system as a whole. From system requirements we need to derive architecturally relevant requirements that drive architecture and design. That is the architect's job.
  • A Functional requirement defines functionality a system is supposed to provide, sometimes also known as capability. Use cases are functional requirements. They define what the user expects from the system on a black-box level. User stories according to Wikipedia denote software system requirements formulated as one or two sentences in the everyday or business language of the user.
  • A Non-functional requirement (also known as quality requirement, quality attribute or "ility") is constraining a solution. There are two subcategories here: operational requirements as the name suggests constrain the runtime behavior of the system (e.g., security, performance) while developmental requirements constrain the design from a developmental perspective (e.g., maintainability, modifiability). As operational requirements often have a systemic approach to a system they are subject to architecture design and therefore are also called strategic requirements. Tactical requirements such as modifiability mostly have only local scope.
  • Constraints - what a surprise - are constraining the solution. A constraint might be technical (we need to use Windows 7 as our OS), organizational (subsystem A should be developed at location A), standard & law (we must abilde to FDA rules), business-related (target costs should not exceed xxx$).
It is the job of a software architect to base all decisions on the forces of the project. Still she/he has sufficient freedom to make architecture and design a complex task. Architecture is about communication and courage to decide.

Monday, July 20, 2009

No future for architects?

I am a big fan of Star Trek, especially of The Original Series and The Next Generation. From a technology viewpoint, it is amazing how all of these technologies such as warp drives or beaming have influenced science. Did you know that the inventor of the mobile phone had also Star Trek in mind?

However, there is one sad aspect. What about Software Engineering? Whenever software plays a role in any of the episodes, you’ll recognize Spock or Data just coding. Do they ever design? There is also no software engineering team in the Enterprise although software should be an important asset in the system architecture. I am missing a chief engineer for software engineering aspects!

Does the computer program itself? Will UML disappear in the future? Or is it just software engineering being too boring for SciFi fans? Interestingly, there are also seem to be no restrooms in the spaceships. This is what they have in common with software engineers :-)

Thus, we could ask how software engineering especially the discipline of software architects will evolve in the future? Will we still use a successor of UML in hundred years? How could the role of software architects evolve?

The best approach to predict the future is identifying space of improvement. It is very likely that such areas will be addressed. How should the ideal process of software design look like? What is the software architect supposed to know and to do?

In most SciFi movies automatization/Model-Driven Software Development and AI are often the answer. Architects express their intent, smart systems then try clarify open issues, and eventually the result is generated.

Another approach could be an organization like the Borgs who strongly follow an agile approach with code ownership, collective programming and test-first.

Any opinions?

Keep in mind: Resistance is futile!