The Extensibility Syndrome

In one of the Facebook groups I am member of there is currently a discussion going on how to deal with extensible design, especially how to prove to a client that a system meets its extensibility expectations.

Focus on extensibility is important but can also be a big hazard, in particular, when engineers overdo it. For example, consider the application of the Strategy pattern. Basically, applying the Strategy patterns implies some kind of uncertainty which algorithm actually should be used. If many Strategy pattern instances are used, this almost always means that the engineers have no clue in which direction the system should be extensible. Of course, the same holds for overuse of other extensibility patterns as well. Such systems become a nightmare for developers and users.

Thus, make clear with requirements engineering or customers what kind of extensibility they really need.

What needs to be extended:

• Should an algorithm be exchanged?
• Is it a whole subsystem or layer that is subject to change or extensibility?
• Is it a component that needs to be changed or extended?

When will it be extended?

• Now,
• Mandatory in the future,
• Optionally in the future?

Which binding time?

• source code,
• compile time,
• link time,
• runtime,
• maintenance time?

Who will be in charge of extending?

• operator,
• user,
• developer

You should only consider those extensions that are mandatory, none of the possible or optional ones that may or may not appear in the future.

In the beginning of software design introduce change scenarios as proposed by the book Software Architecture in Practice, 2nd edition (Bass, Clements, Kazman). Here you’ll learn how to describe modifiability scenarios using scenario diagrams and rating them using so-called utility trees.

You may also use design tactics diagrams that deal with how to implement the different kinds of extensibility mentioned above.

If you follow the principles of my Onion Model, then each architecture design step will only be driven by requirements and their priorities. This way, you can establish requirements traceability within your architectural design.

For extensibility and change scenarios, a Commonality/Variability analysis can be leveraged, especially when dealing with a wide variety of extensions such as in platform or software product line development.

In contrast to common believe extensibility or change is only applicable after you’ve designed those parts of your architecture that are subject to extension or change. You can either extend some functional entities or operational qualities such as scalability mechanisms. This implies that all extensibility and change design activities follow after functional and operational design and not before.

When opening your functional & operational design for extensions or change always use the Open/Close principle to prevent unwanted side effects.

To prove that a software architecture is extensible such as expected by the customer you got different means.

• An end-user or operator might like to see the implemented change scenarios and how they actually work in the implementation.
• Another engineer might like to see how the architecture and implementation support extensions.
• A requirements engineer may like to see the mapping from requirements to architecture decisions.

Modifiability aspects (e.g., extensibility and changeability) are much more difficult than they seem at first place. When the amount of extensibility is overwhelming, you’ll often find wizards or configuration automatisms to hide the mess underneath (which does not imply that wizards or configurators are indicators for bad design).

As Heraklitus once said: panta rhei – everything changes. Thus, be prepared!

Does Cloud Computing have an Architectural Impact?

Let’s suppose you have good reasons to leverage a Cloud infrastructure. Will this decision hava any impact on your architecture? The answer is quite easy: It depends :-)

• If you just use virtualization to exactly simulate a physical environment like in Amazon EC2, then your applications will be oblivious to the Cloud infrastructure.
• If you leverage SaaS (Software as a Service) like Salesforce.com then it’s a SEP (Somebody Else’s Problem).

However, if you use PaaS (Platform as a Service) or IaaS (Infrastructure as a Service) for your applications, then things definitely will change.

It is not so much the domain-specific logic that is influenced, but infrastructural and non-functional requirements will experience an impact.

• In a Cloud environment different storage means are supported. In general, you’ll find Blobs and NoSQL databases. Designing the persistence infrastructure must take these issues into account unless you are still relying on a mainstream DBMS running somewhere in the Cloud.
• For messaging Message Queues are available. Thus, message-oriented communication between Cloud-aware subsystems is a reasonable approach for connecting islands with each other. Often, SOA style communication protocols such as RESTful communication or SOAP/WS-* are supported. These protocols alone imply different architecture paradigms - designing the communication and distribution infrastructure can be a challenge.
• All operational qualities such as performance, availability, scalability, fault-tolerance need to be implemented with the SLAs in mind the Cloud provider offers. If, for example, availability zones are supported in the Cloud you better plan how to leverage them according to your needs. In a public cloud security is even a more critical issue. You cannot naively trust the Cloud provider to keep your mission-critical data secret. Thus, you have to provide means like encryption almost everywhere. For performance reasons, you might want to consider the Cloud as a Grid such as in Apache HADOOP. But how can you partition your system in a suitable way. Is Mapreduce what you need or, otherwise, does your application require a different kind of approach? There are many different concurrency architectures like Master/Slave, Pipes & Filters, Concurrent Reactor that could be appropriate. In contrast to typical application development you are much more constrained when addressing these operational qualities, because many decisions the Clound infrastructure has already made for you. It is YOU who needs to integrate with the Cloud, not vice-versa.
• Developmental qualities such as modularity, modifiability, maintainability can be hard to achieve in such an environment. First of all, you should introduce some kind of governance approach. Secondly, you need to modularize in such a way that governance issues live in harmony with the kind of modularization propagated by the Cloud, in particular by the PaaS APIs. For command & control purposes you need to think about how to administer the Cloud applications. There might be 3rd party products for this purpose. It, however, might also be the case you have to implement your own tooling.
• If, for some reason, you need to use different cloud infrastructures, interoperability might be a big issue given that today’s approaches almost inevitably lead to vendor-lock-in.

Not always, you will be able to start with a green field project, but rather need to deploy an existing application ecosystem or parts of it to the Cloud infrastructure. Migration is not straightforward as we already saw earlier. You may need to combine reengineering and refactoring activities for this purpose.

This posting describes only the tip of the iceberg. I hope, people will develop some best practices (patterns) for how to leverage existing Cloud platforms. I’d be rather sceptical if someone would come up with such patterns soon. You surely remember, patterns must be independently used in three different applications to make them patterns. I cannot imagine, we already obtained that much experience in practice.

So, if you are going to design for Amazon EC2, Google Web Engine or Microsoft Azure don’t forget: Mind the Cloud!

Requirements traceability – The Holy Grail

It often occurs that I am supposed to review an architecture document – that’s one important part of my job. After having read the document, I always try to summarize what I just read. And in many cases, this proves to be rather difficult. One of the reasons is that the subsequent parts of the document seem to be only loosely connected such as graphical models popping up without textual explanation or architectural & technical decisions appearing without any hint what the design rationale could be. However, a reader who is dependent on rumors, insider knowledge, and vague assumptions will see the hodgepodge of syntax but never fully understand the semantics behind this syntax. If the reader is a tester, requirements engineer, fellow architect, or developer such situations need to be considered harmful. Yes I can see the answer is 42, but was the question respectively problem that lead to this result?

The big secret is requirements traceability. Each and every architectural decision must be strongly derived from forces (i.e., requirements and risks). This also helps to keep the architecture simple, expressive and balanced, because this way we can get rid of all these design pearls smart developers and architects typically invent.

From the forces elicitated at the beginning of a project architects derive architecturally relevant requirements. This includes requirements clarification and priorization. Remember: Garbage-in-Garbage-out. If the initial assumptions are wrong, the resulting implementation can under no circumstances meet the expectations.  Of course, we do not expect requirements to be fully available at project startup. We have become extraordinary agile wizards, anyway, and can also handle requirements dropping in. Needless to say, we can not gurantee the same perfect software & system architecture we would have come up with if all these requirements had been fully available from day 1.

As we already got those long-term experience, we are using the onion model.

• For all use cases (part of user requirements) we are deriving a black box view of our system. From this descriptions we also obtain actors and required/provided interfaces. As we see, this step is driven by requirements.
• The domain model is another force. It is a means to understand typical entities in the underlying domain as well as their relationships and interactions. They will turn the black-box view into a white box view. Now you can start to map the use cases to sequence diagrams which is also requirements driven. All of a sudden the domain-specific subsystems, components, interactions, interfaces are disclosed refining our external model. Eric Evans is an excellent source how to deal with such domain-driven design.
• So far we have focused on the domain entities. It is time to add infrastructural requirements such as communication, persistence and concurrency infrastructures as well as operational requirements such as performance, safety, availability. This we do by starting with the highest priorities- that’s why priorization is so essential – and ending with the lowest priorities. Each architecture decision is strongly rooted in requirements.
• The same is done with the mandatory developmental requirements like modifiability, maintainability and then with the optional developmental requirements.

All these decisions are depicted in diagrams, but also explained in the textual descriptions.

What are the benefits of such an approach:

• We get requirements traceability: Firstly, we know which requirements lead to which architectural decisions, thus also obtaining a mechanism to check whether we have already covered all the necessary requirements in our design. Secondly, we know which effects an architectural change such as a refactoring will have on requirements, thus preventing or at least lowering the risk of accidental complexity. Thirdly, all decisions are made explicit. Implicit parts lurk like invisble ghosts deep in the guts of the architecture hurting us when we are least prepared.
• We are driven by priorities. If we miss our milestones, we skip the less important requirements instead of the really important things.
• Readers of our documents will not only understand the “what” but also the “why” making it much easier to get their buy-in. This is what we call design rationale.
• If we additionally come up with leading guidelines and principles for cross-cutting concerns in the beginning we will increase internal qualities such as symmetry, orthogonality, simplicity.

The journey is the reward in software architecture. Don’t tell readers about the target to aim for, but also about the way how you get there. On each junction, explain why you prefer one over the other road.

Writing of such documents can be such a fun. And reading them even more. For example, read Dave Cutler’s book about the creation of Windows NT.

A architecture document shall contain guidelines and design rationale, but no labyrinths.