The hard way to Distribution and Concurrency

The Web is hot. So is Cloud Computing. And Multicores might even become hotter. Whenever there are additional capabilities, software developers will immediately start to exploit them. Why else do we always require new hardware for new operating system versions?

In particular, dealing with distribution and concurrency is pretty simple. Just start a thread in Java or .NET, and the force will be instantaneously with you. Or click on some menu items of your favorite tools to distribute functionality elegantly across the network. 

Unfortunately, many projects fail in providing distributed or concurrent functionality. How comes all these nice programming platforms and libraries don’t suffice?

And the answer of course is software architecture. Especially for cross-cutting concerns such as distribution and concurrency the programming level offers the wrong abstraction layer. It is not sufficient to address concurrency programmatically. Foremost, we must address the architectural layer, then the design aspects before thinking about programming idioms.

Let me investigate concurrency as an example. For this purpose I’ll consider different domains with respect to concurrency:

• In home automation systems there is a large variability of different services with various durations. Think about switching lights on/off, or controlling the climate system. Typical home automation systems are reactive systems that use a central component for event notification and event handling. As many events might occur in parallel,  the Concurrent Reactor Pattern offers the right architectural approach.
• For an image processing system we need a completely different architecture. Here, image data is created, processed and transmitted between subsequent components.  Hence, the Pipes & Filters architecture pattern with concurrent filters fits nicely into the domain.
• When dealing with control systems or Web Servers with asynchronous events arriving all the time that must be handled in a synchronous way, a Half Sync / Half Async architecture provides a message queue which decouples the synchronous from the asynchronous world.
• In Servers that process single messages such as Log Servers, a Leader/Followers patterns applies perfectly. The leader is in charge of receiving the next event. If it receives an event, it promotes one thread in the followers chain to the next leader, while becoming itself a worker that after work completion will become a follower. A nice example for a self-organizing infrastructure.
• In a warehouse system we’d like various actors to store or load items at the same time. Thus, the warehouse core cannot only provide one single interface. Instead, multiple interfaces operate on the warehouse core logic. They can synchronize their data by using a Half Object plus Protocol pattern. 
• In a mobile network or an AI system multiple, concurrent agents may need to coordinate their activities to achieve a common goal. For such systems, patterns like the Blackboard pattern are applicable.
• For divide-and-conquer processing the Master/Slave pattern helps structuring our Map/Reduce problem to sets of interacting components, some of them responsible for the Map, most of them responsible for the Reduce functionality.

We got various example domains, all of which had to use a different concurrency architecture approach due to their diversity of forces such as existence of sessions, variety of services, synchronization necessities, and many others.

Our first conclusion is: different problem contexts require different concurrency architectures. There can’t be a single concurrency architecture for all problems. In addition, we need to deal with architectural issues before introducing finer grained design aspects.

For the next step of refinement it is essential to solve issues at the design level. Think about Half Sync / Half Async where we need to determine  how to organize the synchronous worker threads of the Half Sync layer and how to let them access the shared queue of incoming events in a coordinated way, to address only a few aspects.

Therefore, we need to ask questions like:

• How will we manage shared resources such as threads or connections? Thread pools and other design tactics come into my mind.
• How do we efficiently deal with sharing and locking of data? We might use monitor objects or Software Transactional Memory approaches.
• How can concurrent entities communicate with each other and coordinate each other? Actor-based programming  patterns are one option here.
• How can we efficiently access data? Think about caching, eager evaluation and lazy evaluation?
• Is there a way to efficiently handle I/O? Asynchronous processing such as Proactors, Active Objects, or Asynchronous Completion Tokens come to our rescue.

Basically, all of those design aspects are on the component level. Fortunately, there are many design patterns that help us in implementing this abstraction level.

Our second conclusion: whenever we conduct fine grained design (i.e, tactical design), design patterns and design tactics offer guidance. 

Only in the final step, we are going to leverage all programming language and library features. Eventually, we end up using methods for thread creation and management, mutexes, agents, semaphores, and all those other idiomatic features modern programming platforms provide.

In the best case, a DSL and/or a library could help increasing the abstraction layer of the implementation by hiding details of the underlying hardware system. Think about OpenMP, PLINQ, Akka STM, and many other solutions.

What we learn from this example is that cross-cutting concerns like infrastructure or quality attributes require a sound fundamental architectural approach which is refined using design tactics and design patterns, and then implemented with the available tools offered by the programming platform. For instance, creating a concurrent solution requires an architecture-first instead of an implementation-first approach. For the same reason, SOA approaches favor interface-first over implementation-first styles.

Sure, programming might be much more enjoyable and exciting than drawing architectural boxes and lines, but at the end ad-hoc implementation of cross-cutting concerns causes more harm than fun. And don’t forget: creating a sound software architecture is sexy – at least for software engineers.