The Scope of Product Line Engineering

In Software Product Line Engineering the first activity comprises scoping which basically means to anticipate all upcoming products that should be in the scope. I was asked by another engineer whether this implies that we need to know all applications in advance which will be part of the product line and if yes, what if the number of applications would be VERY large. So, what does scoping actually require? Scoping does not mean that you really know everything in the beginning. Otherwise it would be very easy to define the commonalities and variabilities. As always, architects have to deal with vagueness and uncertainty in this context. My understandimg of scoping is as follows. We need to have an idea of the products in the program family we are going to build. Thus, the first step would be to define use cases / scenarios that are typically expected by an application in the program family. If use cases differ or if they appear in some applications but not in others they will help discovering variabilities. If they reveal common (generic) parts or are identical for all applications they will reveal the commonalities. But how do we obtain those use cases? Either, the domain itself inherently defines such use cases. Take ATMs as an example. Or, we could take a representative sample of applications from which we derive the use cases.
This brings me to another important point. I was also asked whether product lines are always a simple partitioning in domain engineering and application engineering. No, they are not. Take car manufacturing as an example which many consider as the prototype of Product Line Enginering. If a car type is manufactured as part of a product family, it will contain common parts all members of the family share as well as variabilities such as different colors, chassis, and so forth. Thus, we have domain engineering where we produce the common core assets and application engineering where we build and customize the individual cars. So far, so good. If we think this a little bit further, some of the assets we use will themselves be parts of other product lines. Maybe, the supplier of the entertainment and navigation system will also leverage product line engineering. Or maybe, the car manufacturer will provide other assembly lines where core parts such as the doors of the car are produced in a product line approach. In other words, a product line can be a combination of different product lines intertwined with each other in a pipeline.
But it is also possible to find the other way round? Suppose, we are going to build a generic web shop application for different customers. We are setting up two separate product lines, one for SOHO and another one for enterprise customers. After a while we find that both product lines can be based upon an additional product line where the common parts for the SOHO as well as for the enterprise web shop are developed. In the SOHO and enterprise product lines we are then just adding the deltas. Obviously, such a setting only makes sense if the SOHO and Enterprise Web shop solutions reveal a lot of differences. Otherwise we would better establish one single product line and handle the differences of both shop variants by variabilities. By the way, finding the appropriate approach is also an important outcome of scoping.
In one of my next postings I will address the issue of why SOA engineering can be considered Product Line Engineering. So, stay tuned!

Generative Software Architecture

When designing really huge and dynamic software and system infrastructures such as the Internet or cellular networks it is very unlikely to come up with a complete structural design in the beginning. You cannot simply draw some architectural viewpoints that cover those systems as a whole. The reason for this is straightforward: such systems are designed to grow all the time and in unanticipated ways. On one hand these systems represent product line architectures. On the other hand they cannot be treated as such in terms of engineering because they have virtually infinite scope. But how can we successfully design and implement systems of that scale or complexity? You might have encountered the term Ultra Large Scale Systems (ULS) in the past months which exactly deals with this problem. As I already explained in another posting a ULS is a system of systems as Linda Northrop and her peers pointed out. An appropriate metaphor often mentioned in this context is building cities. So far, we have built software systems that resemble building single houses or city blocks. In the next step, we are going to build complete cities and even countries. Complexity represents one of the core problems in developing such large systems, because inherent complexity won't disappear. Hence a possible solution must master complexity instead of getting rid of it. The key strategy as in many other disciplines is emergent behavior. In other words: How can we partition our large problem into a set of simple problems and ways to combine these problems to an overall solution? In the world of software architecture this could basically mean different things depending on the scale we encounter:

• we could define the basic infrastructure and let complex parts be integrated into this infrastructure
• we could define a set of rules to dynamically build and evolve such an infrastructure as well decribe integration constraints for parts
  interacting with and over the infrastructure or
• we could combine the aforementioned approaches by defining a small set of rules and smaller predefined infrastructural parts

Those rules are not only axiomatic rules as in a language grammar, but also define constraints and requirements for the parts that are integrated into the infrastructure. In such an approach, the non-functional properties must be derived from those rules and constraints.
They cannot be evaluated in advance for the reasons already introduced. Obviuosly, to cope with future challenges, such a system must also provide extension rules and hooks to even adapt the infrastructure to changing requirements.If you think about it: Why could we come up with a technology such as the Internet with all of its complexity? Internet technologies already
fulfil the properties mentioned above. The Internet consists of smaller infrastructures, e.g., Internet backbones. Technologies such as IP, DHCP, or DNS define a rather simple (but not simplistic!) rule set for extending and evolving the infrastructure as well as for integrating servers and clients into the infrastructure. All those very simple technologies are combined to a highly evolvable and dynamic, large-scale system which is open for extension. Basically, HTTP and the Web are only one of those extensions. As a real-life example take the brain which consists of rather simple constituents and combinations between those constituents, but reveals the most complex emergent behavior we have ever dealt with. From my viewpoint, we should direct our research activities to those generative architectures, especially when dealing with large-scale software systems. Emergent behavior is the key to master complexity in such systems.

Non-functional requirements

Many software projects don't fail because of functional aspects. In general, experts know very well how to build a system in their domain. The number one technical reason for failure are non-functional requirements. Issues such as reliability, availability or performance have to be addressed in addition to all the domain logic. Unfortunately, many domain experts are not necessarily experts in these non-functional topics. And the efforts to get consistency or availability right can easily outnumber those for functionality. Ask what the CTO of Amazon thinks about this challenge. Thus, my first rule could be described as "always define key developers for the requirements with the highest risks and relevance". These key developers are in charge of teams that explicitely address one or a small number of these issues. The size of these teams and the number of teams depends on your overall project size.
Another important point in this context is desribed as requirements elicitation and requirements engineering. It is interesting how unusable many requirement specifications are. Statements such as "System should provide flexibility" are worthless. What exactly do these stakeholders mean by flexibility? Which parts of the systems should be flexible and when? Of course, you are also doomed to fail if you get a bunch of requirements with no priorization. The same application would look completely different in case security is the number one requirement compared to the same application when performance had the highest priority. So, my second rule simply says "Architects and key developers are in charge to clarify all requirements including their priorities. For each pair of requirements, it should be clear which one has precedence in concrete design decisions". As many requirements might contradict each other, this is inevitable to survive. If your management does not agree to detail requirements or prioritize them, either refuse to be in the project or, more realistic, document this fact and possible implications so that all stakeholders know they might encounter project failure. Unfortunately, non-functional requirements are incredibly diverse. Performance is totally different to security. Even security might mean a lot of various things such as authentication, authorization, confidentiality, name it. Thus, my rule 3 is: "Detail all non-functional requirements in the beginning of the project, maybe establish checklists, and use these information to clarify requirements".
Developmental requirements such as flexibility (e.g., changeability) have to be addressed early only if you are going to design a product line where a Commonality/Variability analysis is of foremost importance. If this is not your primary setting, developmental issues should be treated as a tactical refinement of your software architecture. Thus, the strategic base line architecture is primary influenced by the domain logic and by operational requirements. The integration of developmental issues is a further refinement activity. Considering issues such as extensibility in early phases typically leads to generic and inperformant architectures with instantiations of patterns such as Observer, Interceptor or Strategy. In addition, your job as an architect tends to continue forever in such environments. This is where I'd like to add my fourth rule "Domain logic dominates infrastructure; infrastructure dominates operational requirements; operational requirements dominate developmental requirements". Of course, it is also important to specify when operational requirements appear in the running system (run-time, install time, compile time, ...) and who is in charge to inject them (operator, user, developer, ...).
As of the huge diversity of requirements it is not easy to come up with general rules of thumb for all of them. Some observations might be interesting in tbis context: A given requirement might be invasive (i.e., cross cutting through other parts of your system) or non-invasive (i.e., it is sufficient to add some additional infrastructural components to the architecture). The same requirement might be invasive for one system (security in all its aspects must be supported everywhere) or non-invasive (we just can add a firewall and don't have to care about security at all). A requirement might me measurable such as the response time to a method request or not (such as removability). Requirements that can be measured can be covered by testing, all other requirements must be covered by architecture and code analysis. Last, but not least, my rule 5: "Classify your requirements and provide strategies how to deal with them in architecture design activities".

Of course the overall goal of software engineering must be to

• Let engineers focus on their domain by strong separation of domain aspects from other aspects.
• Automate where possible and useful.

There are several efforts to support these goals such as Model-Driven-Software Development, Product Line Engineering, Aspect-Oriented Software Development, SOA.

One way could be to provide DSLs for the domain, additional domains for the most important non-functional requirements, a technology that easily lets project teams integrate these DSLs under one umbrella (e.g., by leveraging AOSD techniques, Semantic Integration). A Product Line Engineering approach could define core assets which become then part of platforms or frameworks. With SOA, platform dependencies could be overcome. Is this just Science Fiction or a reasonable future vision? Let's work on this :-;

Back in Town

If you wondered why I didn't post anything in the last weeks. End of February I had some nice days in La Palma. Then I went to the Netherlands. First to Amsterdam as a tourist. And after that I completed a personal underground project. In the last years I had secretly worked on my Ph.D. thesis with Prof. Dr. Jan Bosch being my esteemed promotor (Rjiksuniversiteit Groningen). The reading committee consisted of Prof. Doug Schmdt (Vanderbilt University), Prof. Hasselbring (University of Oldenburg) and Prof Engels (University of Paderborn). If you got curious: In the dissertation I addressed the issue of analyzing and understanding software architecture using patterns. As the core domain I investigated remoting middleware and SOA. This is basically what I did since the early Nineties when we wrote the first volume in the POSA book series. That time I had been the only expert (at least to my knowledge) who worked on that topic. In the dissertation I could prove that pattern systems help express domains such as remoting middleware and paradigms such as SOA idiomatically. This way, you can also leverage patterns to evolve these domains and paradigms as well as define best practices for application developers. Thus, patterns are not only applicable for forward engineering but also for backward engineering. In the RuG Groningen, it is allowed to provide a Ph.D. thesis that contains several articles which can be co-authored by other people if you can prove that the promovendus had contributed most (and significant) parts in the majority of the articles. All those articles must have been accepted by respected magazines (in my case IEEE, ACM, Springer) or conferences (in my case PLoP, EuroPLoP) which means they already participated in a review or shepherding process. This kind of approach is helpful for someone like me who must investigate and write a dissertation in addition to his regular work.
If you like to read the Ph.D., then obtain a PDF version from the university library. Since the defense ceremony was very impressing (in an old-fashioned and nice setting), you can also look at my flickr photo set. By the way, Groningen is a lovely town in the Netherlands. You should definitely visit it whenever you are in the area. In addition, all Dutch people I met are extremely nice. I am glad that everything is done although I am already missing the great co-operation with Jan. Now I am ready again for future blog postings.

Simple is Beautiful

One of the most important architectural qualities is simplicity. If an architecture is simple, it is easy to understand, change, or apply. Simplicity is not necessarily the opposite of complexity, because complexity is often an inherent property of problem space, while simplicity refers to the fact how to efficiently cover problems in the problem space by concepts in the solution space. However, simplicity might help mastering complexity. The secret to simplicity in software architecture as in all other systems is to define core architectural elements which are simple, but can be combined in various ways resulting in a broad spectrum of emergent behavior. The human brain is an example for this fact. While the core elements of a brain such as the neurons and synapses are surprisingly simple, their combination might result in incredible complex interactions, basically so complex that we don't really understand how the brain works. Same for peer-to-peer systems or ant populations.

The fundamental properties of of these examples are as follows:

• The entities and their interactions are simple
• If any of these entities and interactions are removed, full coverage of the problem space is not possible anymore
• The same design principles are applied orthogonally throughout the system. Such design principles cover functional as well as non-functional properties
• The system design provides atomic components on top of which higher abstractions (e.g., subsystems) are built, thus resulting in a hierarchical instead of a flat structure

If you think about the properties above, you'll see that a violation of one of them immediately results in a non-simple (i.e., complex or simplistic) system. For instance, you might have simple entities and relations, but when the system consists of a giantic flat structure, you might be lost. Trying to understand Windows Vista by looking at its assembly code, simply does not work. Simple systems offer what I call architectural beauty. Just by using a small set of easy-to-understand concepts, you can even cover complex problems in a domain.

What architects tend to forget is the fact that humans are not able to consider and understand more than 7 entities at the same time.

Thus, upcoming Ultra Large Scale systems will only succeed if they rely on a simple fundament consisting only of a few core concepts that can be combined to a broad spectrum of emergent behavior.

Simple does not mean simplistic. A system is simplistic when its core elements are not capable of covering the whole problem space.

How can we achieve simplicity? Simple software architectures often reveal a high pattern coverage, use abstraction concepts such as components or services, and use the same solutions for the same problem contexts. Evolving a simple system might happen in two ways, by evolving the core concepts or by applying the available abstraction mechanisms.

By the way, in case you wonder how to check whether an architecture is simple: one test for simplicity could be the following. An architecture is simple when is can be explained to other experts in a few minutes without using visualizations.

It is important to mention that in a software project it is not sufficient to deal with simplity in the software architecture. Simplicity should also be the leading principle for other aspects such as the process, the tools, the technologies, the programming languages, the models, the DSLs, and so forth. Also keep in mind that it is generally insufficient if each of these aspects is simple. You might use a simple but powerful language such as Lisp and will be completely doomed to fail when trying to build a multi-tier enterprise system using Lisp even if the architecture is simple. This implies, that the combination of simple aspects must be simple itself. Therefore, consider simplicity a holistic approach.

Mind the "Keep it simple, stupid" principle to make your professional life simpler but not simplistic :-)

The Singleton Patterns and The Concept of Identity

During my Java EE 5 tutorial at the TOOP 2007 conference, an attendee asked how the Singleton pattern should be best leveraged in Java EE. In my opinion the Singleton pattern is more a workaround than a proven solution and thus it is seldomly recommendable to apply it.
Basically, what the Singleton does is to guarantee the existence of one single instance of a class.

Example from JavaEE:

// get ConnectionFactory via JNDI
ConnectionFactory cf = ... // single instance
Connection c = cf.CreateConnection(...);

The factory uses the Singleton pattern such as in:

class ConnectionFactory {
static private ConnectionFactory _instance = null;
protected ConnectionFactory() { ... };
static public ConnectionFactory instance() {
if (null == _instance)
_instance = new ConnectionFactory();
return _instance;
}
}

For sake of brevity I didn't care about thread safety. For our purpose you can just ignore this fact. In the code you will recognize that no external user of the class can access the protected constructor so that the only way to obtain an instance is calling the static factory method instance().
This way, there will be exactly one instance whenever one or more clients access the class.

There are many problems with this approach. The most important one is that a Singleton defines a global variable such as ConnectionFactory.instance().

Suppose, we need exactly one object instance. Do we really need to rely on Singleton?
Let me introduce a simple example.


class ThePrinter {
private String printerAddress;
public void print(String fileName) { ... }
public ThePrinter() { printerAddress = MY_CONSTANT_VALUE; }
}

In the class above, we can provide multiple instances and each of these instances will semantically define the same object (thus we should provide appropriate equals and hashCode contracts).

From an identity viewpoint, we need to constrain our class to provide objects with the same identity. The way a Singleton tries to guarantee this is by only providing one single instance. But it is legal and much more appropriate (in terms of OO pureness) to achieve the same result using an immutable class as shown in the printer example above.

Another option of course would be if we need to constrain the numbers or kinds of instances created by a class such as in a Manager or Pool class. These only allow creating specific instances or a maximum number of instances. Thread and connection pools are typical examples for such behavior. In this context, a Singleton is a Pool with capacity == 1. I won't show you more code here. You can google for Peter Sommerlad's Manager pattern.

Another "inverse" kind of approach in terms of identity is the following. Suppose, you got thousands of users on your Web site. You started with providing one volatile user object for each online user which made your system less fault-tolerant and more resource consuming. How could you change this? In other words, how could you provide a limited number of instances of user objects (# user objects << # online users)?
a) implement a pool of fixed user objects that wait for incoming requests (Active Object pattern)
b) when a request comes in, it is enqueued
c) one of the idle objects dequeues the request
d) it retrieves the user id from the request and looks up the user's data in the database
e) it operates on the user data and stores it back to the database
f) the result is returned to the client
g) the user object is ready to take another request and thus another identity

This approach does even work if multiple user objects run under the same user identity. In this context, each pool object can temporarily take any identity because the identity's data itself is persisted somewhere else. Thus, identity is not the same as instance.

Keep this in mind when you next time feel addicted to solve an identity problem with the Singleton pattern or other weak approaches.

It is surprising how often architects and engineers forget to think about identity and how to design and implement it.

Architecture Refactoring

Refactoring as introduced by Martin Fowler is an excellent practice for bottom-up and structure-preserving gardening activities in your implementation. Refactorings are documented in a canonical form that resembles pattern templates to some extent. A typical example is Extract Interface. Assume, you've designed otherwise unrelated classes, each of them implementing the methods store() and load() for serialization. To treat these classes polymorpically, extract the common methods to a common interface, for instance IStorable that provides these methods and is implemented by the aforementioned classes. In an agile setting, refactoring denotes an essential tool, because the incremental integration of new use-cases keeps the tactical architecture aspects in flux which means that implementation artifacts require continuous modifications. You should "embrace change" as Kent Beck once said instead of trying to circumvent it - which, as we know, is impossible.

As you might remember, Joshua Kerievsky came up with "Refactoring To Patterns" in 2004. The basic idea behind his excellent book is to identify places in an implementation that use proprietary solutions for problems which should be better solved using patterns. I also want to give you an example for this approach. Suppose, you have implemented a class ShareValues that contains a list of consumers. Whenever a share value changes, you iterate through the list and call event handler methods on each of these objects. While simple to implement, the flip side of this approach is the tight coupling unnecessarily introduced. As soon as new consumers are added, we have to manually modify the ShareValues's list of consumers. Obviously, applying the Observer pattern comes to our rescue here. The advantage of refactoring to patterns is that we are able to introduce a higher abstraction level.

If you think that even further, we enter the world of what I call Architecture Refactoring. In Architecture Refactoring we refactor the architecture itself.

Possible Examples:

• Partition Responsibilities: If a component or subsystem got too many responsibilities, partition the component or subsystem into multiple parts, each of which with semantically related functionality.
• Extract Service: If a subsystem does not provide any interfaces to its environment but is subject of external integration, extract service interface.
• Introduce decoupling layer: If components directly depend on system details, introduce decoupling layer(s).
• Rename Entity: If entities got unintuitive names, introduce appropriate naming scheme.
• Break Cycle: When encountering a cycle on subsystem level, break it.
• Merge functionality: If there is broad coheshion between two modules, merge them.
• Orthogonalize: If two parts of an architecture introduce different solutions for the same problem, choose one preferred solution and eliminate the other.
• Introduce strict layering: If in a layered system, a layer accesses lower layers without necessity (relaxed layering), enforce strict layering.
• Introduce hierarchies: If several entities are only variants of a particular entity, introduce a hierarchy.
• Introduce Interceptor hooks: If we have to open an architecture for out-of-band functionality according to the Open/Close principle interceptors should be introduced.
• Eliminate dependencies by dependency injection: Reduce direct and wide-spread dependencies of Parts in a Whole/Part setting by introducing a central runtime component (Whole´) that centralizes dependency handling with dependency injection.
  

So far, these are only suggestions of potential candidates. But you can see the possibilites now. Obviously, we should collect a whole catalog of such architecture refactorings. Note, that higher-level refactorings might incur lower-level refactorings the same way architecture patterns often refer to design patterns. Extract Service will very likely lead to an Extract Interface refactoring if the implementation already exists. Orthogonalize is heavily leveraging refactoring to patterns.

From my viewpoint, we have just started to understand and apply refactorings in a more holistic context. As the old saying goes, we have just seen the tip of the iceberg. Architecture Refactorings will guide architects to identify potential problems in a software architecture and also provide them with refactorings to solve those issues.

How to start with requirements

Often when I am involved in software development projects as a software architecture consultant, team members are wondering how to start in terms of requirements engineering and how to proceed once all the requirements are available. As this is an issue that is critical for project success, I will elaborate on it in this and upcoming postings. Note however, I won't cover tools such as DOORS, or IBM/Rational. Here is my short take on this, tools in a nutshell if you wish so. Michael's rule of tool usage: Tools are relevant iff they give more support than they incur trouble :-;

So, let us assume we are starting a new project. For simplicity, suppose we are in a company that produces healtcare products. Some senior managers in our company found out that it is time to build a web-based framework for physicians where patients can obtain information, ask for prescriptions online and arrange meeting schedules. Product management asked several customers for theirs required and desired features and it turned out that customers would buy such kind of product. Thus, we got our business case.

As a software architect, the first step is to interview stakeholders about their requirements. The problem here is that senior managers and product managers are often far away from software architecture and technology. Hence, it is our task to clarify requirements with them and translate management level requirements to technical requirements. For example, I often hear sentences such as "this must be performant and scalable and secure and flexible." Yes, but what exactly do they mean by "flexible" or by "performant"? What if both requirements contradict each other? Another issue here is that different stakeholders often have different perspectives. A project manager in a multi-site software development project might be more interested to partition subsystems in such a way that each subsystem is developed by only one site, while a software architect thinks more problem-space oriented. Political issues might also be lurking such as sites which are striving for being responsible for specific parts of the product. Obviously, outsourcing and offshoring denote important factors as well. All of these issues need to be addressed by software architects. In order to deal with them, architects must disclose the relevant issues from day one. Here, it turns out how important communication skills are for architects if you ever doubted it.
My first recommendation is: software architects should arrange meetings where they interview all relevant stakeholders. Thus, identify stakeholders with senior management. Then take the time to interview all of these stakeholders. Prepare yourself with checklists where you list questions you want to ask the stakeholders. Otherwise, it might be a problem to get full coverage or to obtain comparable answers. Note, that these checklists differ depending on the personal backgrounds of interviewees. It is a common rule that senior managers only seldomly can give technology or architecture advice (at least not explicitely). In the interviews clarify requirements. Ask interviewees what they exactly mean with the requirements. You should also be prepared to ask more concrete questions and guide the interviews. For example: our online patient system framework should be flexible. What does flexible mean - parts can be removed, added, exchanged, removed or does it refer to modularity? How fast must changes be committed? How many changes are anticipated? Can these changes be added at runtime or at compile time or at maintenance time? What kind of changes do typically occur? Don't forget to ask interviewees about the priorities they give these requirements. If they tell you, all requirements are of identical importance, ask them questions like: "If you had to choose one of these requirements as the most importan one what would it be?" or "What are the features that must be available in the beginning and what do you consider the highest risk?".
After the interviews set up a kick off meeting where all architects and stakeholders are invited. Present your findings to them and also prepare a rough proposal of the project setup (milestones, work packages, activities, requuirements with priorities and risks). After the meeting all the attendees should commit to a final project plan, have a common understandings of terms, and what exactly the requirements mean, and what their risks and priorities are.
In this context, it is important to mention that there are different types of requirements:

• Functional requirements define the concrete domain-specific problem (in terms of the problem space not of the solution space!). Here a DSL is a good start. Or at least, should be one of the results of domain modelling.
• Operational requirements such as performance or fault tolerance relate to the operation of the final product.
• Developmental requirements define architectural issues such as flexibility.

Customers are often more interested in functional and operational qualities. These qualities will also have a huge impact on the strategic core of your product. Developmental qualities are mostly tactical issues. They are of high relevance for the development team, especially when we are going to build a product line such as in our example project.

Often, it helps a lot when defining the boundaries of the system to be built using context diagrams. Additionally, a use case analysis with participation by customers is essential.

It is important to mention that a software product line such as the one introduced in our running example requires special focus on commonalities and variabilities of the products we are going to build. For example, some of our customers might need the patient meetings to be synchronized with Exchange Server, while others might use IBM/Lotus, or even exotic platforms. Thus, our patient framework must be built in a way that allows this kind of variability. Another variability may be the different looks and feels of the Web sites.

After you got final agreement and committment on the requirements you can start to proceed to the next step and set up the project which won't be covered in my postings. However, it turns out to be important to asign important subsystems to key developers and also assign the most important non-functional issues also to key developers. Software architects are then responsible for the whole picture, but key developers for their own topic of concern. In the beginning, architects and (and key developers) will design a base line architecture and communicate this architecture to the stakeholders. The design should reflect functional aspects but also at least the core non-functional concerns.

In our example, security and usability will be most likely the most important requirements. Thus, we should organize two teams, one responsible for security and the other one for usability.

In the whole project activities is essential to ensure requirement traceability. Each design or implementation decision should be motivated by requirements and follow the architectural baseline. This is where architects must act as supervisors.

This concludes the first posting in my "series".

Architect's Toolset - CRC Cards

Whenever I am involved in a software architecture design, CRC cards are one of my favourite tools for team discussions. CRC is not the abbreviation of Cyclic Redundancy Check in this context as you might assume. It rather stands for "Class, Responsibility, Collaboration".
I was always wondering why I still can find developers not familiar with this tool.

So, instead of telling you all the theory, let us start with an example. In the simplistic example above, I like to design a Web Shop. After the engineers defined the most important use cases and scenarios they are now in the middle of determining all core entities. For example, they are playing around with the "customer buys items from catalog" use case. Of course, using a UML tool could be a solution but did you ever see more than two people sitting in front of a screen working efficiently on a problem? This simply does not work. Here, CRC cards come into play. A CRC card is a simple sheet of paper containing three separate sections (see figure above):

• The class name is the name of the component being described. In the example, we are introducing a Shopping Cart.
• The responsibility section contains all responsibilitiesfor which this component is in charge such as adding or removing items in our example.
• In the collaboration section we define on which other components our component depends, the so-called collaborators. In the example, the shopping cart must be able to co-operate with the product catalog.

All of this is simply drawn on a sheet of paper and then visibly put on a white board, poster, whatever is available. After a while the board will be filled up with CRC cards. In order to make the collaborations visible I often use a string for connecting the CRC card to its collaborator(s) using pins.

Because this is a flexible, paper-based solution it is easy to change CRC cards, remove them again, add new ones, add or remove responsibilities or collaborators. Everyone in the team is able to participate in the whole brain storming process.

Obviously, in an agile setting the CRC cards will be incomplete in the beginning and then keep growing over time.

Later on in the design process we might add additional information to the cards such as superclasses.

There is a lot more to say about CRC cards. Ward Cunningham , the "father" of CRC cards, and Kent Beck provided an excellent paper for OOPSLA 1989 that you should definitely read (http://c2.com/doc/oopsla89/paper.html).

Product Lines

Software Product Line Engineering is one of the most interesting areas in software architecture today. And it is increasingly important in software development, in particular for the development of standard software products and embedded systems - those that we ususally consider to be software-intensive. Now, you might ask, what the hell is a software product line. One of the commonly accepted comes from the Software Engineering Institute (CMU) which defines a software product line (SPL) as “a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way”. Got it? Let me introduce an example. Suppose, you are going to develop new IDEs (Integrated Development Environment). These new IDEs should come in different flavors, one edition for Ruby, one for Fortress, one for C#. How could we manage to develop such a series of editions? First approach: We develop all editions separately. This turns out to be a nightmare as we will need to set up a project for each language edition. But even worse, after a time, we will recognize that all of these projects will develop very similar modules, a GUI, a debugger, a source code browser, a project and configuration management subsystem, a refactoring toolset, an internationalization component, a plug-in manager, and so forth. Many of these core components are programming language independent. So why should we constantly violate the DRY (Don't Repeat Yourself) principle?
Ok, what is the second approach? In our second attempt, we start developing the Ruby IDE taking special consideration to implement all subsystems such as the GUI, the debugger, you name it, in a re-usable way. After the Ruby IDE is completed, we can re-use existing components for further IDEs. Basically, we got a set of independent lego units which we are just might compose in different ways. Obviously, this is a much better approach in terms of productivity through re-use, but it turns out to add other challenges. For example, we have to come up with different application architectures that use the existing components in a bottom-up way. All of these architecture require a lot of effort themselves. In addition, it turns out that some of the components developed for the Ruby IDE are not appropriate for the Fortress IDE. Thus, we need to refactor them to be more generally applicable. But after that work we need also to refactor the Ruby IDE to integrate these refactored components. Of course, we might choose to end up with different versions of components. This is exactly one of the reasons why some projects experience death by unsystematic re-use.
Ok, what is the third approach? In the third approach we set up a development team that first analyzes which concrete IDEs we are going to develop. The team will take all requirements and define an general architectural framework for all of these IDEs. It will come up with commonalities which are all assets that should be part of the general framework. E.g., components such as the project and configuration management subsystem that are basically identical no matter which programming language you need to support. The team will also define the variabilities , i.e., things that vary among different IDE editions. For example, the abstract syntax editor framework might be a commonality, but the different concrete syntax editor implementations will vary. As we expect users to be interested in adding their own components to the IDE (for example tools that support programming conventions), we integrate a plug-in manager. This is an example for a component that supports variation over time. So the general team will set up an architectural framework with explicit variation points as well as other commonly used assets. As we expect these assets and even the architecture framework to evolve, we need to set up an environment where common assets and their evolvement can be supported. By the way, assets in this context can be different kinds of things: implementation artifacts, documents, models, test plans, and so forth. The setup of such a project will be very expensive but the Return On Investment will rock. As soon as everything is in place, the Ruby IDE team will be able to systematically instantiate the architectural framework with all commonalities, and also bind the variabilities according to their own needs. They will save a lot of time because they don't have to come up with their own, new software architecture from scratch and they can even use existing assets. What we have defined here, is Software Product Line Engineering. All products instantiated from the product line are often called Software Program Families. I hope, you got the basic idea. It is like in other industries such as car manufacturing where the car design team comes up with a new series of cars, defines what parts all of the models could share, and also defines the variabilities. Obviously, a development team must carefully identify whether is worthwhile to set up product line engineering. It is probably a bad idea when you only have two different products.
So far, the teams that develop concrete products use the architectural framework and the common assets but the rest will be plain-vanilla software engineering. In most cases, there is a lot of space for automating many activities. Take a production line in a car manufacturing plant as an example. In Software Factories the idea is to define a systematic plan of how a product is configured or implemented using the common architectural framework and common assets. The Software Factory will then allow automating the process of developing a concrete product by leveraging model-driven software development. In order to make this experience less challenging, the idea is to come up with a domain-specific language (instead of forcing everyone to use UML) and provide a generator that includes all knowledge about the architectural frameworks, as well as about commonalities and variabilities. The generator will take a model as an input and generate (most parts of) the concrete product.
An issue we must consider in this context is how to deal with evolution and change of core assets It is unlikely the architectural framework or other assets will remain unchanged forever. Thus, a systematic approach is required to give feedback from product instantiations to the product line itself. This also implies, you should always have separate teams for the product line itself and for the products. Otherwise, organizational issues might me very challenging. In other words: even, if you provide a product line engineering approach, you might be doomed to fail due to organizational problems. Needless to say, that the development process is also significantly different which you should take into consideration.
An interesting issue is the differentiation between a platform and a product line. In many cases, the most important core asset of a product line will be a complete platform. Eclipse is a good example for this. In this case, the platform will be the critical part of your whole product line. If it suffers from design erosion (see my last blog posting), this will have an impact on ALL products. That's why systematic evolution of common assets is so essential. Note, that a software product line does not necessarily have to introduce a platform. This is only one extreme, where we have prefabricated implementation of the common architecture and other common subsystems and components. Another extreme might be an application framework. Or we might only have a set of loosely-coupled artifacts and a software architecture. Thus, there are lot of options to implement software product lines.
In future postings I will address architectural issues related to product lines in much more detail.

Design Erosion

Most software systems start with a clean and comprehensible software architecture. But then, developers are often forced to add or change the system under increasing time pressure. After a while the system is polluted with workarounds. The strategic core architecture becomes vague or even completely lost. Design erosion has caused a breakdown of the original software architecture. Is this the normal way of an architect's life or did we simply fail? In the waterfall model design erosion will be inevitably caused by architects trying to cover all requirements at once, which we call big bang approach. As you know, in a sufficiently large system you can never anticipate all requirements. Thus, you can never come up with a sound architecture in the first place. From my viewpoint, this problem illustrates why agile processes are not an alternative but a must. In order to cope with these challenges, we have to embrace change. Change will definitely happen. Hence, we as architects must open our software systems to allow tactical extensions. Using additional use cases or when priorities are changing, then even the strategic design might be subject to change. Refactoring will hence be an important tool for architects. As implementation and architectural soundness must go hand in hand, applications such as Sotograph will enable architects to make sure that the implementation does not violate architectural guidelines and conventions as well as quality aspects. But this is only the tip of the ice berg. Testing, organisation issues, and many other aspects play an important role. It is possible to prevent design erosion, if your process model, organization, methods, and tools are appropriate. In product lines this factors are even more important because of the significantly larger impact of any change. In some circumstances, however, requirement changes caused by new business models, technologies or desired features might be too far-reaching. In these cases, it is sometimes much more effective to throw away the old system and build a new one from scratch. "Throwing away" does not mean that you should throw away your well-proven best practices or experiences. Just ged rid of the old software system, not of your expertise. As a consequence, a good project will enforce the documentation of best practices (e.g., domain models or DSLs, patterns, guidelines). Design erosion is also a good example that all those comparisons of software engineering with other disciplines are often like comparing apples and oranges. For instance, when building a house no one would ever come up with proposals such as adding additional rooms or floors after construction is completed. But people tend to think software engineering should support exactly that kind of flexibility. That is also the reason why software engineering, especially designing a software architecture is a really tough task.

Teaching the Architects

I published the last posting several weeks ago. But, of course, I have a good excuse: In the meantime, I gave a seminar on Software Architecture and participated in several Siemens-internal events. But now the time has come to continue with my architecture blog.
This time I'd like to address one specific issue. How can we effectively teach other persons everything they need to know about designing a high quality software architecture? For the sake of brevity, I won't cover what a software architect should know in this posting. In the seminars Frank (Buschmann) and I give, we mostly offer a mixture of Powerpoint presentations, discussions, and group exercises. In the presentation sections we also address things we've learned in projects. In the group exercise 3-5 attendees are then asked to design a small example application such as a Web Store or Chat Server. This setting works very nicely, but is far from being sufficient. Thus, another and additional approach is to teach and consult small project teams in real-world projects. In this scenario we start with a series of seminars and then participate in real-world projects as architecture consultants. However, this isn't sufficient too as we also need a kind of certification for software architects. Of course, small companies might not afford to spend all that time and budget for educating their staff in architecture concerns. But that is another topic.
I'd like to know what you think. I'd also like to know about your experiences. What did work for you and what didn't? How did you become a software architect? I am looking forward to all comments.

Ultra Large Scale Systems (ULS)

In the last OOPSLA Linda Northrop gave a talk on Ultra-Large-Scale systems (ULS). ULS introduces a lot of unprecedented challenges. An ULS is basically a system of systems each of large scale itself. Consider the Internet as one possible example, especially when you combine the Internet with Pervasive Computing. There are a lot of issues to be taken into account for such system. One example is the human interaction which will definitely be part of those systems. Linda motivated the problem we face with such systems with the metaphor of building cities instead of building buildings as we do today. How can architects cope with such systems? From my viewpoint, the development off ULS systems will be driven by emergent behavior and a mixture of system/software architecture efforts. By emergent behavior I address the problem that it is simply impossible to anticipate all networked entitities, especially when we consider continuous change. Thus, we can't come up with a fixed architecture. We are entering completely new areas here. There is no chance to really foresee now all the implications of an ULS on software architecture. However, we will need to introduce software/system architecture principles and guidelines that should impact the architecture. In addition, we will need to come up with some core components such as repositories, persistence stores, other kinds of central services that are essential. Everything would be initially driven by typical workflows that should happen in those systems. Essentially, we will need to start simple. Use simple abstractions and principles that can be combined to more powerful structures. The human brain is an interesting example as its main constituents are rather "simple" while its overall behavior (i.e., emergent behavior) is incredibly complex. One should keep in mind that it took mother nature millions of years to develop those brains by constantly modifying, enhancing the design, even removing old design that didn't work, until today and beyond. That implies, that we won't get it right in the beginning. Thus, ULS development will definitely need experimenting and playing around with concepts and configurations.
Swarm intelligence was a sort of precursor in that direction. Define behavior of entities as well as their interactions locally and then let the system act globally. The killer application were ant populations.

Architect always implements

This night I had a dream. I was supposed to have a surgery. Eventually, the physician came to the room. I heard him whispering to a nurse, giving her a document. It was a list of instructions he had written down. Oh, no! The nurse was going to be the surgeon!
Sounds like a nightmare. But isn't it something at least some "software architects" should be familiar with? A group of architects is specifying a software architecture. Once finished they throw it over the fence to the developers, and dissapear. I know that the comparison is like comparing apples and oranges. However, there is some truth in it.
In my last tutorial on software architecture Scott Meyers asked me why I recommend the "architect always implements" principle. And that is exactly the point. The task of a software architect starts early in the process and spans maintenance and evolution. After the strategic core design is complete, architects help to refine this core to obtain the more concrete, tactical design. And software architects always need to supervise the implementation as well as decide for tools and technologies. Find more details in my former postings.
If you consider these tasks, a software architect is only capable of fulfilling these tasks when she/he gets some implementation practice. Another issue is "eat your own dog food". You as an architect should make sure your designs work as expected by helping with their implementation. That also gives you more credibility with respect to other project members. And the other point is: how could you possibly recommend tools and technologies without practical experience? Would you trust a software architect who did not implement for the last ten years and has no clue about UML, middleware, C++/Java/.NET, Eclipse, Visual Studio, databases, Subversion, Clearcase, Maven, ... ?
Does that mean you need to implement a lot? Definitely not. It rather implies, you should implement some smaller parts that are not on the critical path because you will still have a lot of other work such as communication with stakeholders, refactorings, meetings, refinements, you name it.
Of course, it also depends on project size. In a small project with only 5 people, you cannot afford that an architect does not implement. In a large scale project with maybe hundreds of people involved , it might not be feasible to implement anything, at least not for the lead architect. However, if you're a lead architect, you should keep yourself knowledgeable by applying tools, technologies in small toy projects, and by participating in design reviews and code reviews. Of course, a software architect as I described in a former posting, is not born as such but will start as an excellent developer and then may turn into a software architect after a couple of years. Thus, she/he should like implementing, anyway.
In the projects I've been involved "architect always implements" has proven to be an excellent principle. That's the reason why I am wholeheartedly recommending it.

Architect's Project Diary

Have you ever been in a meeting where participants discussed a topic which already had been discussed several times before but no one was sure what the concrete decision had been. If yes, didn't you consider that recurring and endless discussions a total waste of your time? To be honest, this is something you will also experience in your family life. But isn't there something we can do to avoid these unnecessary discussions? As software development projects are filled with meetings, the propability of such discussions is tremendously higher which is a good force and motivation to think about countermeasures. What I consider extremely helpful, is a kind of project diary. Within a team, whenever you are deciding on important issues or on issues which you know might raise further discussions, you should document these decisions using a project diary. Mention who was involved in the decision process, what alternatives had beed discussed, which ones were ruled out and which ones accepted. Explain the "what" but also the "why" in this context. Obviously, it doesn't make sense to document everything with all details. Fundamental decisions, however, need appropriate documentation in your diary. You might structure your diary simply as a list of meeting minutes or structure it using topic areas, whatever fits your needs. The advantage of such a diary is that you can understand decisions even a long time later and explain to people why exactly that decision had been taken. Or the other way round: If your requirements or your project context have changed and you need to change previous decisions, using the diary you might get a clue which decisions must then be "refactored". Anyway, a diary is absolutely helpful, when done appropriately and without unnecessary details.

Live from OOPSLA 2006 (continued)

I will be less verbose in this second part of my travel report as I am summing up almost the whole OOPSLA from my personal viewpoint (it is impossible for a single human person to attend all those interesting things in such a large conference). Thus, my impressions are a little bit biased as they only reflect my personal research interests. And I am a lazy writer :-) So don't expect me to give you every detail. You should definitely attend OOPSLA yourself next time.

I met a lot of various friends and people during the conference days (that is, as some say, the most important topic here). And as Terri Parr (conference chair) proposed I met a lot of new people. That was a really exciting experience. One of the funny stories is that I met a professor and it turned out that he was Axel Schreiner, one of the famous german computer scientists who left Germany and now lives in the US (I read his books on compilers and many other issues years ago during my university education).

Tuesday:
This is the official start of the main conference. It is the 20th OOPLSA aniversary and it is back there where it started: in Portland, Oregon. This year, OOPSLA is collocated with Dynamic Languages Symposium, GPCE, and PLoP. Terri Parr who is the conference chair announced that there are 1140 attendees, 430 from overseas with 460 first timers. Oregon locals: only 59! This proves that OOPSLA is a true international event.
The program comittee accepted 26 out of 157 submissions with 5-7 reviewers per paper.
2 essays out of 9 were accepted. In the Onward! track 10 submissions of 22 were accepted and 2 will appear in the proceedings. Terri told the audience in her greeting talk that Siteseer rates OOPSLA as one of the 50 most influential conferences.
The first keynote (Onward!) by Brenda Laurel (California College of The Arts) was on designed animism. Basically, it was a little bit esoteric but nonetheless interesting and inspiring. Brenda illustrated that every animal and plant has a "soal". She mapped all those ideas of animism to distributed and pervasive systems. And the basic driver for all of this is having fun. That's a rather short and incomplete explanation, I know. But thinking about it is left to you as an exercise :-)

In the language track (Research Papers) I heard a talk on how to dynamically extend typs by expanders. Instead of inheritance, source code change it is better to use adapters. With tool support this is a powerful feature.

Linda Northhrop (SEI) gave an invited talk on ULS: (Ultra Large Scale System). The bottom line was "Scale changes everything" in terms of software development. Take the Internet as an example with a decentralized, large number of users and data. Other applications include Healthcare Infrastructure, Homeland Security, Networked Automobiles, Military Systems (platforms, sensors, weapons, warfighters). ULS is a research study sponsered by DARPA. Goal: US Information Dominance.

Problem of such systems include continuous changes and sustainability. These systems are much more complex than what we have seen before. Heterogeneity is an additional issue. From the technical challenges, the US Army is concerned that complexity could be beyond reach (what about 1 billion line systems). That's why they initiated the study by the SEI Team and an Expert Panel. Process: Meet with experts in different fields at a 3 day meeting. Result was - as you might have expected from an expert group - a Research Agenda. They took a reductionism approach according to Linda.
First question they asked themselves: when do we encounter an ULS? Possible properties to look at could be:

• size: in lines of code
• number of connections
• perception
• number of processes, interactions
• number of overlapping policy domains
• number of people involved

Expect unprecedented scale of some of these properties in an ULS. ULS will be interdependent webs of systems, people, and other things. These are basically webs of systems each of them revealing internet scale (!!!).

Characteristics of an ULS system:

• decentralized
• inherently conficting, diverse, unknown requirements
• continuous evolution and deployment
• heterogeneous inconsistent changing elements
• erosion of the people/system boundary
• failures are normal
• new paradigms of acquisitions and policy

Today's approach for large systems is to engineer top-down, controlled and use an agile perspective for small projects. For ULS a new perspective is required.

Metaphors to make that more obvious: today we build buildings, ULS is like building cities. Thus, guidance/rules are required. Another metaphor could be to think in ecosystems (independent systems, interactions, ...). ULS can be considered a socio-technical ecosystem with a lot of competition for resources and varying policies settings. In ULS we deal with ecosystems made of people, software, hardware, governments. Hence, engineering cannot be the metaphor. There are many conflicts with our approaches today. ULS == decentralized which is in conflict with today's centralized approach. Requirements in ULS are mostly unknown while today we believe that requirements can be known in advance and they do only change slowly. One of the additional tradeoffs will be stable continuous evolution: today we think of system improvements in intervals. In an ULS we face huge heterogeneity while in today's software systems we believe that effect can be predicted, configuration is accurate and in control. Social interaction is completely not considered today.

Challenges in three areas: design and evolution, orchestration and control, monitoring and assessment.

To investigate in ULS we need to establish interdisciplinary teams, learn from existing projects. Game theory might be a good point as well as the consideration of other existing networks (biology, body, ...) .

The ULS team came up with 7 areas of research

• Human Interaction: "understanding users and their contexts"
• Computational Emergence: digital evolution, metaheuristics in software engineering, algorithmic approaches in software engineering
• Design: on all levels: software, rules, regulations, .... (the whole ecosystem)
• Computational engineering: more expressive languages, other kinds of modularity, formal techniques
• Adaptive System Infrastructure: decentralized configuration and evolution
• Adaptable and predictable system quality: qualities - what do they mean in an ULS context
• Policy, Acquisition, Management
  

Find the study as pdf in http://www.sei.cmu.edu/uls. A collaborative network is beginning to develop.

After the keynote people geve the feedback there was a little too much US military proximity, but that the topic is highly relevant.

There has been a Panel on the same issue with people such as Doug Schmidt, Gregor Kiczales, Linda Northrop. Here are only some excerpts from the panel to give you an impression.

• Gregor, Doug: First we need to solve problems of small systems before moving to the ULS.
• Kevin: Software as a problem must be solved in general.
• Martin who is focusing on monitoring and repairing systems: uncoordinated growth require repair and monitoring support. Resources must be available to enable growth.
• Ricardo: Let us play with such systems to find out before actually deploy them.
• Peter: Lot of lessons we have not learned.
• Neil Harrison (not a participant of the panel) cited Conway who said that organisations influence ULS. Look at your organization and that's how your systems will look like.
• Peter: Deficiencies show lack of communication in teams. Conway's law is fundamental.
• Doug: ULS is about heterogeneity. Can be challenging when parties involved are not really in favor of cooperating. Important: Building cities instead of buildings. Design process is important. Architecture consists more of rules. We need more abstractions.
• Gregor: Ambiguous Meaning. Now we have not even a common understanding of most terminology.
• An attendee from the audience asked about precursors of ULS. Doug thinks, current systems like military systems are precursors. A system that integrates all critical government systems could be a precursor. Linda mentioned, Internet+Wireless (devices) combined could be a precursor.
• Another attendee asked about metaphors the team came up with? Cities, Ecosystem, Human interaction were the ones. The panel was a little bit inert and difficult as most participants are more technology biased. Sociology has many good points as Doug mentioned.

At Tuesday evening there was the most emotional part of OOSPLA. IBM invited to a dessert buffet and a memorial to the life of John Vlissided who died so early in late 2005. Desserts were John's favourites if you may wonder. Interestingly, the last time I had personally met him on the last day of OOPSLA 2004 in Vancouver we had talked about - cookies. If you don't know John: he was one of the Gang of Four authors of the seminal book on design patterns. Erich Gamma, Ralph Johnson, and Richard Helm (the other authors) talked in the evening event about their memories working on the book project with John. They also showed some video excerpts which really revealed to the audience what smart and nice guy John has been. My personal recommendation. Search for John Vlissides in Google and learn more about him.

Wednesday:
A Keynote on A Growable Language given by Guy L. Steele introduced Fortress. See http://research.sun.com/projects/plrg/ for more details. Fortress design goal was: to do for Fortran what Java did for C. Guy who was involved in Java design mentioned, there were many proposals to add scientific computing features which were rejected to keep the language small and portable. Thus, funded by US DARPA IPTO (High Productivity Computing Systems Program), Guy and his team started a new language design from scratch. Goal: availability by 2010 and beyond. Similar work is done by IBM (e.g. X10). Fortress offers accelerated development even for desktops (Multicore). Key idea of Fortress: - don't build a language - grow it. E.g., many people would like to add a lot of different primitive types. But no way! That's not the way Fortress will work. Whenever possible, Fortress' approach is to integrate a library instead. Hence, library designers need control over syntax and semantics (not just over method calls). Fortress offers a few primitives: binary words of different sizes, linear sequences, heap sequences, user defined parametrized types, user defined polymorphic operators. The compiler leverages aggressive type reference aggressive static and dynamic optimization. Libraries provide types such as lists, sets, matrices integers, floats ... with physical units. Data structures might be local or distributed. Fortress also intends to ease parallelism. In addition, it contains a programming notation that is much closer to Math. Instead of having to deal with Unicode characters, Fortress offers an ASCII (Wiki-like) notation. Example: A UNION {1,2,3,4} .
The strategy of the language specification team consists of studying existing applications, studying how libraries can improve coding, and adding language features. Replaceable components in the langauge system avoid a monolithic standard library and encourage change. Fortress tries to make abstraction efficient, The type system basically consists of objects and traits (like interfaces, may contain code). Multiple inheritance of code not fields is supported as well as multiple inheritance of contracts and tests (automated unit testing). Traits and methods may be parametrized. Primitive types are "first- class data and control models". To implement Fortress more flexibility is required than just compiler optimization. Things such as transactional memory (atomic blocks, ...) will be relevant. Libraries may define which operators have infix, prefix precedence, whether juxtaposition is meaningful, what operators mean.
Parallelism in Fortress was not a goal but a pragmatic compromise. Parallel Programming is still difficult and error-prone. The question the Fortress designers asked themselves: can we encapsulate it in libraries? They did it to some extent but also added some implementation. Loops are parallel by default in Fortress (library convention) .

Charles Simonyi presented Intentional Software Corp tool in Onward! that allows to combine different DSLs and use an IDE to have different perspectives on the same source code. Very interesting and inspiring talk. The tool is currently evaluated by Cap Gemini and ThoughtWorks. Basically it allows to you view your code as code and also have some other perspectives at the same time. For example, if you implement a state machine, state diagrams could be made visible in your IDE editor. This is an excellent idea that can not be described in a textual blog. Expect more from this cool company in the future.

Another invited talk was by Joshua Bloch (Google, formerly Sun) whom many of you may know as the author of the Effective Java book. His topic was about how to design a good API and why it matters?

APIs are important assets for companies but might also be a liability. The Process of API Design basically consists of

• Gather Requirements with a healthy degree of skepticism
• Start with a one page document
• Write to your API early and often
• Writing to SPI even more important
• Maintain realistic expectations.

General Principles that are useful in this context:

• APIs should do one thing and do it well (also important here: use comprehensive names)
• API should be small as possible but no smaller.
• Implementation shouldn't impact API. Minimize accessibility of everything.
• Names matter - API is a (little) language.
• Documentation matters- document religiously.
• Consider performance consequences of API design decisions.
• API hast to co-exist peacefully with the platform.

In terms of class design Joshua gave the following advice:

• minimize mutability
• subclass only when it makes sense
• design and document for inheritance or else prohibit it

Method design is another point he mentioned:

• don't make the client do nothing the module could do (otherwise lot of boilerplate code is the result)
• don't violate the principle of least astonishment
• fail fast - report errors as soon as possible after they occur
• provide programmatic access to all data available in string form (e.g. stacktrace as one string vs. everything as set of elements)
• overload with care
• use appropriate parameter and return types
• use consistent parameter ordering across methods
• avoid long parameter lists
• avoid return values that demand exceptional processing

Exception design is another important matter:

• throw ecxceptions to indicate exceptional conditions
• favor unchecked exceptions
• capture information in exceptions.
  
  

On thursday I gave my two tutorials (Software Architecture - Strategies, Qualities, Principles as well as SOA from an Architectural Viewpoint). Both were well crowded and I got a lot of positive personal feedback after the tutorials. Neil Harrison and Scott Meyers both attended my Software Architecture tutorial and asked smart questions (as I had expected it). I will put PDFs of these tutorials on my Web site. Thus, I don't go into the details here. The most interesting event of the day was a fire alarm during my first tutorial. There was a power outeage in the part of Portland where the venue was located. Hence, I made a reading session out of my tutorial while the beamer didn't work. Guess, that's at least something the attendees will remember :-;

The next day I left back to Munich. I am really looking forward to the next year where OOPSLA will be located in Montreal.

Live from OOPSLA 2006

This is the third day of my OOPLSA/GPCE 2006 experience. This year, OOPSLA takes place in the Oregon Convention Center in Portland.
I spent half of the time talking to other people such as Rob van den Berg, Arno Zimmermann, Eric Mejier, Jimmy Nilsson, Neil Harrison, Peter Sommerlad, Doug Schmidt, Arno Haase, Markus Völter. Conferences are an ideal place for meeting other interesting people.
Yesterday, I attended a tutorial by Arno Haase and Markus Völter on Domain Specific Languages and Model-Based Development. As tool chain they used OpenArchitectureWare and Eclipse. As an running example Arno and Markus illustrated the domain of state machines. The tutorial was absolutely entertaining. Now, I know about all of the important buzzwords. Just kidding :-) Indeed, the main benefit of the tutorial was its pragmatic approach. Funny to see Markus torturing Eclipse to get the most out of it.
Today, I planned to see Niclas Nilsson in his tutorial how to write code generators. Unfortunately, the tutorial was already crowded with people. Tutorial speakers like me can attend other tutorials free of charge if there are some seats left. And that can be a challenge. As an alternative, I intended to participate in a tutorial on programming the Sun SPOTS robots. Guess, what? Thus, I eventually attended a tutorial by Jeff Garland and Richard Anthony, both experienced practitioners for building large scale systems. They talked about building solid distributed enterprise software architectures. In details, they focused how UML and architectural principles can help for this purpose. It was the intent of the tutorial to illustrate how to effectively use UML and architectural principles to design and document an architecture. I missed a little bit the architecture principles aspect. All in all, I can recommend the tutorial.
This evening, I will meet Eric Evans and Jimmy Nilsson for dinner. Before, there will be the Welcome Reception, the first event in a series of social events.

Active Web - formerly was Web 2.0

I am track chair of a track on Web 2.0 for the upcoming OOP conference (http://www.oopconference.com). In this context, I also agreed to give an introduction talk on what Web 2.0 is all about. Simple task, I thought in the beginning. Well, Web 2.0 is about social networking and AJAX and tag clouds and YouTube and Digg and podcasts, and ... , you name it. The name Web 2.0 is really confusing, isn't it. But, what does Web 2.0 really mean?
From my viewpoint, there is a big commonality among all those concepts and sites. Everything in Web 2.0 is about (inter)activity.
In AJAX and similar technologies - the web page is not passive anymore. Instead, it contains code fragments that actively pull/push information from/to backend servers to achieve a better user experience.
In social networks users are not passive consumers anymore, but actively connect with each other. To enable this, web sites must actively connect different people with each other.
In pages such as Flickr, YouTube, Digg, Amazon people actively share information and media. Podcasts and VideoPodcasts allow people to become active content providers. Same for P2P networks, Wikis, Blogs, ....
In other words, Web 2.0 should be better renamed to reflect the aforementioned observations. It is about the evolution of the Web from a passive medium where we find a small group of active content providers and a large group of passive content consumers to a new Web where the boundaries between content providers and content consumers more or less disappear. This new Web provides technologies and means for Web users to switch from a passive consumer to an active provider almost immediately. It is a Web where everyone can participate. It is no longer a technology platform dedicated to large companies for marketing purposes or establishing Web-shops. It is more about social interactions and active participation. It is an open medium for everyone.
My suggestion thus is: let us rename Web 2.0 to Active Web. I think, this name much better reflects what the new Web evolution is all about.

Architectural Beauty

Whenever we are enjoying music, literature or paintings we associate some kind of beauty with these art works. Physicists and mathematicians consider some theories as incredibly beautiful and elegant, for example the theory of relativity or quantum physics. Architecture of buildings can also be beautiful. And what about software architects? Do we consider some software architectures as beautiful? Well, the answer is not really surprising. Of course, everyone of us from time to time encounters a software system, he or she feels very confortable with. Two questions immediately arise: Are there some objective properties a software architecture must or should reveal so that we consider it as beautiful? And if that is the case, is architectural beauty an important quality aspect or is it just a kind of preception without further value?
For me personally properties such as the following ones are important so that I consider a software architecture as beautiful and elegant:

• 
  Simplicity: A software architecture should be as simple as possible but obviously not simplistic. If an architecture reveals unnecessary complexity, it is almost impossible to capture the strategies and tactical design behind the architecture. This is strongly coupled with readability. To check whether a software architecture is simple, ask the chief architect to call a person who has never seen the architecture before. The architect should then in no more than 5 minutes explain the architecture to the said person. If the person can grasp the fundamental architectural idea within that time, then the architecture should be simple. Another related issue in this context is expressiveness. An architecture that is expressive implements a (domain) model consistently. Hence, it is very easy to find the key domain entities and use cases in the architecture. I am emphasizing the domain model, because some (ugly) software architectures tend to mix implementation issues with domain entities which makes them not expressive at all. Separation of concerns is one of the means to achieve expressiveness.
  


• 
  Orthogonality or conceptual integrity means that within a particular software architecture the same solution is applied to address the same problem (context). This is where patterns come in: using the same patterns for the same problem contexts is important. Likewise, it is essential not to unnecessarily reinvent the wheel again and again. This is also sometimes mentioned as the concept of least surprise. Of course, orthogonality is not constrained to patterns. For instance, a software architecure lacks orthogonality if you find different kinds of error handling strategies or memory management strategies all over the system. Note, that orthogonality is closely related to simplicity and expressiveness.
  


• 
  Correctness: Even, an expressive and orthogonal system might be plain wrong. If you like to get a spreadsheet application from a development team, you won't be very happy as a user if you receive a text processor instead. Correctness is not limited to functional aspects. A system that does the right thing but takes an incredible amount of time for each activity is also considered incorrect from a user perspective. Correctness might be improved by re-use. Re-using well-proven components or designs instead of inventing your own stuff is obviously helpful.
  


• 
  Symmetry is an important topic as well. There are two kinds of symmetry, structural and functional symmetry. Structural symmetry is tightly coupled with conceptual integrity (see above). Functional integrity means for eaxmple: when there is an open-method there should also be a close-method. Breaking of symmetry in this context may lead to incorrect systems. Kevlin Henney used functional symmetry to illustrate why he considered the GoF's factory pattern as incomplete. In the factory pattern there is a create-method that helps to hide all the complexitites of object creation from an object user.The factory pattern states that it is applicable in all situations where object creation is rather complex. Kevlin argued that when object creation is complex, then object deletion is also complex in most circumstances so that there should also be a delete method in the factory pattern.
  

Of course, this list is far from being complete. I just introduced some points to give you the idea.
There are also some signs when an architecture doesn't reveal architectural beauty. For example, violation of layering or dependency cycles in the design are always signs of severe problems or design erosion which brings me to another point:
Often, architects and developers come up with very beautiful architectural design in the beginning. After some change requests or extensions to the system which are often applied with time pressure, the software architecture erodes and with it the architectural beauty. This means, architectural beauty is not carved in stone but might disappear after a while, if you are not cautious.
All of these properties of software architecture that I consider as preconditions for architectural beauty are also qualities. They help to achieve a specific purpose. Thus, architectural beauty is not independent of architectural quality. Both are only two sides of the same coin.

Live again!

I am back from vacation. When my vacation started weather turned from sunny to rainy. After my return to the office, the sun is back. Must be related to Murphy's Laws.
In my vacation I used the time to increase my knowledge. Well, almost the truth :-) In fact, I spent a lot of time for my sports activities such as running and biking. The rest of the time I was busy because I had to prepare some articles and talks.

• I wrote an article on SCA (Service Component Architecture) and SDO (Service Data Objects). To dig into the details I used the Apache Tuscany M1 implementation for Java. My opinion: SCA and SDO are really cool technologies. They might not be perfect but go into the right direction. In detail, they address composition and modularization aspects of SOA systems as well as ESB issues.
• In addition, I had to dig deeper into .NET 3.0: WCF, (W)WF, WPF, WCS. All these TLAs address really exciting technologies. I am absolutely interested how to combine WCF (Windows Communication Foundation) with SCA.
• Another cool framework I used was Ruby on Rails. I've been a Ruby expert for a long time, but had only very little knowledge about Ruby on Rails. After using it, I am really impressed. This framework proves that a sound language design has a direct impact on what you can build. Same thing with Java and C#: maybe possible, but wouldn't feel natural to developers. It is amazing how fast Ruby on Rails applications can be built.
• I had to prepare and organize the inevitable Web 2.0 track for the next OOP Conference. Web 2.0 is exciting as it combines technologies with advanced user experience. I got Markus Völter for the track and excellent speakers from Google and Microsoft. My talk will introduce the Web 2.0 space.
• For the upcoming JAOO conference I promised a talk on Spring.NET. It is not as powerful and huge as Spring. Nonetheless, it offers a powerful dependency injection container supporting AOP stuff as well as APS.NET, .NET Remoting, Serviced Components, and more. Hopefully, .NET developers will soon recognize the power of IoC Containers which really offers a productivity boost and reduces dependencies on concrete technologies. A wizard for Visual Studio .NET would be great.

As you can easily see, I like playing and experimenting with all these new toys. It helps me in my job but is also great fun. Maybe, this was the only upside of the rainy weather in Munich.