Work Life Balance

This time I like to address a completely different topic which seems to be completely unrelated to software architecture and software engineering, at least if you only scratch the surface. People like you and me working in a software engineering company or department tend to spend a lot of time for their job. When I started my career, project work was so exciting that I often continued to work in my spare time. Guess, what I did during vacation? If someone asked me for assistance or provided me an additional challenge, I kept saying yes to every offer. After some years I experienced total work overload and even got some health problems. I was forced to spend more time for other activities such as sports, arranging music, photography, or literature to recover from these overload situations. At the beginning, I thought and feared, spending more time for such leisure activities would significantly reduce my productivity. But to my surprise I made exactly the opposite experience. Spending more time for non-work related activities, significantly increased my productivity. That is also the reason why extreme programming suggests to limit work time to 40 hours per week. If you ever admired those colleagues who stayed in office for more than 10 hours day by day, you should consider the fact that after a specific amount of work time bug rates increase dramatically and work results tend to reach lowest levels. Thus, working too hard for a long period of time could even have a negative effect. Reminds me of Dilbert's boss who always keeps saying "work smarter not harder". I really enjoy running or biking after work and during weekends. My favourite place is the Isar river in Munich which offers beautiful trails and helps me first forgetting and then successfully addressing all those work-related troubles and worries. If you ever had to cope with a complex problem in a project and couldn't find an appropriate solution despite of your hard efforts, just stop working and go running, biking or what else you prefer. In many cases, this will also help finding an appropriate solution for your project later on. Over the years, I spoke to many famous friends in our software engineering industry and they made exactly the same experience. Most of these well-known experts spend significant time for additional non-work related activities.
If you want to become am accepted expert in your field, take care of an appropriate work-life-balance, or in terms of work activities: sometimes, less is more.

Release It!

A lot of Java and .NET developers are involved in Enterprise and Web development projects. According to official statements by senior management or marketing of software development companies or departments, no one ever seems to have faced major problems in such projects, not to mention catastrophes. However, if you meet other developers at all those software engineering conferences worldwide, you will hear some war stories in personal communications - maybe after a few bottles of beer or wine. According to Henry Petroski "learning from failure" is one of the most important issues in software development. We all have made faults in the projects we were involved. Thus, one way could be to forget those errors in the next project and cause the same problems again and again and .... Do you know the de ja vu feeling when you think in a concrete project situation "why do they make exactly the same mistakes like in previous projects". The other way to address these project failures or problems in a constructive way is to learn from them without trying to identify a scape goat, because attacking people won't be of any help here. It is the whole organization including all stakeholders that should be ready to learn from failure in a positive way. Recently, I read a book on this issue which was written by Michael T. Nygaard and published by "The Pragmatic Programmers" - I really love the books from "The Pragmatic Programmers", to be honest. It is called "Release It! Design and Deploy Production-Ready Software" and covers problems related to software production and deployment. The author illustrates potential risks, he derived from real life projects, as well as countermeasures to cope with these risks. It is exactly the kind of book that is helpful for practioners like me. The book is partioned in four parts: part I deals with stability problems, part II with capacity problems, part III with general issues and finally part IV with operations of software systems. I really enjoyed reading this book and learned a lot. I am too lazy to go into all the details, because it is really better for you to read the book yourself. I'd like to see more books of this kind. Of course, I understand that no company would be willing to share all its failures with the rest of the world, especially with its competitors. But unfortunately, only a few companies have a philosophy of learning from their own failures. Thus, books that abstract such problems from the real projects, and show potential pitfalls and their solutions in a more general way could be a treasure chest for all software engineers. Believe me! Documenting patterns is much easier as they represent proved solutions. No one will complain about documenting patterns for this reason. But I won't give up trying to convince everyone that a culture of learning from failure would help us being more productive and successful. At the end, most technical (r)evolutions are caused by learning from previous solutions that reached their limits. Maybe, someone has read such books and can give some recommendations. More information on the excellent book by Michael T. Nygard is available here.

Product Lines and Platforms

In Software Product Line Engineering one of the core activities deals with determining the central assets common to all programs in the scope of the product line. These assets are not constrained to implementation artifacts but might contain models, aspects, documents, test plans, to name a few. Jan Bosch once proved that ad hoc re-use of assets does not work which leads to the conclusion that re-use of assets in product line engineering requires a more systematic kind of approach. It also implies that re-usable core assets do NOT represent a set of unrelated components, but rather should be combined in a predefined and more tightly coupled aggregation. Now, we are entering the universe of software platforms which denote the central fundament of software product lines. The glue of a platform might consist of different elements such as configuration files or DSLs & generators. It is noteworthy that the platform represents the central core of the software product line including all common components but also supporting the range of variabilites within the program family (BTW, I've already explained in previous postings how architects may analyze commonalities and variabilites.) From the platform all instances of a product line should be derivable by using configuration mechanisms, APIs, a DSL, or other means. If you think, you only seldomly face platforms and product lines, you are plain wrong. The Java JDK and the .NET Framework Classes are examples for such platforms such as all other commonly used APIs. From my viewpoint, all SOA based applications also represent examples of product line engineering.
Why this platform-based approach sounds very reasonable it also reveals some challenges. Suppose, different product line instances, e.g., various mobil phones, leverage a common platform, e.g., a software platform that includes all required communication stacks such as the UI, persistence features, power management, etc. What if one of these application projects requires a modification of one of the common core assets within the platform? There are different alternatives in this context. As precondition, I assume that the modification could not be handled by variation (using a kind of variability hook in the platform), but requires the modification of a common platform component. In a naive world, we would just derive a new branch of the platform without any impact on the other family members. It is a no-brainer that this alternative is unacceptable in the real world as further change requests could emerge each of them leading to a new specialized platform version. Thus, we would end up in a sea of platforms. This is of the same dimension such as asking Microsoft to provide a customized Windows for each customer. Why is this such a big problem? Suppose, that you need to evolve the platform in the future. How can you manage evolution facing a sea of platforms? In the best case, it will work with large efforts and costs, but lead to design erosion forcing the platform team to come up with a complete new platform every second or third product line generation. Needless to say, that the RoI through re-usability which was the primary reason to establish a product line won't be too impressive in such a scenario. Thus, we need another way of handling modifications! Any change request should lead to a modification of the platform. Yes, this will require all application projects to leverage the modified platform. When the platform is a significant part of the application, then we should be ready to pay this price for the sake of platform evolution. In such an approach the platform will be evolved in a systematic way, thus reducing the risk of design erosion. However, successful platform lifecycle engineering is not only a matter of implementation. The development organization must establish a team that is in charge of platform development with a clear interface (and firewall) between the platform team and the application projects which are building the product line members. Otherwise, we will end up in exactly the same approach as illustrated in the first alternative. Of course, a platform team is responsible to meet its customers' requirements. At the same time, a platform team should be empowered to reject all change requests that have a negative impact on the majority of application projects. Moreover, application projects need to synchronize with platform development, and also send change requests and bug reports to the platform team. For these co-operation aspects, an appropriate product line engineering process and tool chain must handle all synchronization and coordination issues. MDSD techniques (Model-Drived Software Development) help to decouple platform from application developers, while increasing productivity. This is the reason why MDA (Model Driven Architecture) introduces PIMs (Platform Independent Models) and PSMs (Platform Specific Models).
To summarize, product lines and platforms denote twins, because product line engineering requires the development of a platform that combines core assets in a common way, while integrating variability management. Unfortunately, platform development is one of the underestimated activities in product line engineering, which is why many projects fail. The reason for this is obvious. Each platform modification has an impact on ALL program family members and is not constrained to a single one-off-application. In addition, the situation gets even worse and messy if you consider the fact that a platform is not only a set of implementation artifacts but also includes tests, documents, models, aspects, among many other entities. Thus, only a systematic platform lifecycle engineering approach will help successfully develop and evolve program families.
If done right, product lines can save time and money. If done wrong, they will make developer life like hell.

Doomed to fail

Suppose, an organization offers you a large amount of money if you agree on the following arrangement. You'll apply for a management position offered by a competiting company that is presumably going to develop a new innovative and software-intensive product. And it will be your task to sabotage their project. I know, that you never ever would accept such an unethical offer. Nonetheless, I am asking you for the moment to think of the unlikely case that you would agree. The main question in this context is how can you manage this task successfully. As you might have expected, I got ten recommendations for you:

• Decide to use a waterfall model. If engineers complain, just tell them that you can't live with the uncertainties and missing predictability of agile approaches. Convince engineers that computer science is a science which implies that they could easily collect all input at once in the beginning, and then systematically design an appropriate software system. If they object to this statement, ask them whether this means they are not able to follow the same kind of approach as other disciplines. Make clear, that from your standpoint agile processes are only for pussies and that you don't respect proponents of this paradigm. If they tell you that agility is about addressing risks, respond to them that you consider agile developers the highest risk in any project. For the same reason, reject any test-first activities from the beginning. As a revenge, tell the development team that in order to reduce budget, multiple engineers will have to share one telephone and one workstation. This is what you understand by Pair Programming.
• Don't introduce any roles. This makes your staff clueless and helps abide to the "Jack of all trades but master of none" approach. People should feel either responsible for everything or for nothing. Using this setting, project members won't be able to ever know whom to address for clarifications. Everyone will be developer and architect. Don't assign the role of a lead architect as this would unnecessarily improve productivity. There is only one predefined role. Make the most unorganized person in your team the project manager.
• Requirements are essential drivers for any project. Thus, a perfect means is to collect a large set of requirements that are as vague as possible ("our application should be flexible"), and in the best case are contradicting ("system should be hard realtime but offer runtime flexibility") or not feasible ("we need 100% availability without any additional costs"). If architects ask you to assign priorities, just tell them that all requirements are equally critical and important. Add to this recipe another dimension by constantly adding, changing or redefining requirements over space and time, especially in later phases of the project. Needless to say, you should never remove any requirements!
• Communication should be leveraged in a proper way. On one hand, try to minimize effective communication between engineers. On the other hand, fill the schedules of your engineers with an infinite sequence of unproductive meetings. These meetings should not have any agenda, be open-ended, mandatory for all engineers, and unstructured. In addition, embrace a culture of e-mail communication with dozens of mails regularly spreading around. Tom DeMarco has labeled this kind of paradigm "Corporate Spam". Obviously, continuous demotivation is also helpful here. Never give developers any positive feedback. Only refer to the incredible and stupid mistakes they made. Introduce the "moron of the month" award for this purpose. Be a control freak and check and discuss every decision in the project, even the unimportant things. This will drive people crazy (e.g., in meetings) and will enlarge your level of joy.
• Outsourcing represents an additional means to worsen the situation. Thus, you should follow an offshoring and outsourcing strategy to play the communication game introduced in the previous discussion. Especially, selecting outsourcing teams with completely different culture is of high importance in this context. Don't let anyone figure out in the project team nor in the outsourcing locations who is in charge of what. One face to the customer won't work very well here, because your goal is to play the game of engineer torture. In the optimal case, language skills of involved persons should be low as otherwise communication would be more effective.
• Force the development team to address all technical challenges with unproven, upcoming technologies where no one in the team could obtain any experiences so far. The application should be based on MDSD, AOSD, Web 2.0. LINQ, WCF, Multicore systems, Semantic Web, REST - add your own favourites here. Tell the people in the team that they are allowed to use whatever tools, technologies, or languages they prefer. This way, the resulting heterogenous tool chain will help you decreasing productivity, while making developers think they got freedom of choice. Another, alternative approach is to force developers to uss inappropriate tools such as the C#-command line compiler, Notepad, Microsoft Word and Visio.
• At the same time don't give people any time for competence ramp-up. Motivate this by telling everyone that you consider "training on the job" the most effective way to learn new technologies. People not capable of learning on the job, are concept-addicted idiots who could be easily substituted by a bunch of chimpanzees typing on a notebook. This behavior will keep morale high. Note: I am refering to YOUR morale here.
• Architecture should be addressed inappropriately. In most cases it is sufficient to prescribe the exclusive usage of UML. As another option consider to develop dozens of different DSLs for even the smallest subdomains. Enjoy when engineers then try to come up with integrated solutions. Architecture documents should have no specific structure, never include all relevant parts, only motivate the "what" and never the rationale. Don't allow any domain modeling during start-up to make sure that stakeholders DO have a DIFFERENT understanding of the underlying domain (concepts). Never introduce architecture guidelines for crosscutting concerns such as fault management, security, or logging. Don't assign the responsibility for subsystems or such cross-cutting concerns to single individuals. This strategy supports mixing domain-specific functionality with infrastructural and non-functional issues which is a constant source of all kinds of errors (-; Applying patterns should be considered a tabu as all other kinds of best practices. To be even more effective, introduce anti-patterns and pretend they are patterns. Last but not least, don't prescribe any locations where to put all those documents. Recall your mantra of IT sadism "Searching is better than finding".
• Introduce re-use by copy-and-paste only. Argue that you are expecting developers to think in terms of open source and that from your point of view this implies a copy-and-paste philosophy. In this context, it would be a mistake to enforce any configuration management tooling. After a while, the application code will be flooded by re-used code fragements that - in case of luck - were changed in the meantime. As soon as an error pops up in only one of these fragments, all developers will feel the pain. Re-using binary components should be considered a bad idea as it will constrain your control. Moreover, enforce and motivate your team to develop everything from scratch. Don't trust any 3rd party middleware, tools, operating systems, but let developers develop these things. Using this approach, the project will focus more on infrastructure than on domain which supports your sabotage stategy.
• Make your system a SOA system. Come up with a great number of potential integration problems and distribute the command to solve all these issues with loose coupling, i.e. using a service-oriented architecture. Invite consultants from market analysts and other companies to tell your team about emerging business and technology directions. Your team will then fall into the SOA trap, forced to consider topics such as Governance, BPM, SOX, EAI, BAM, ESB, REST, WSDL, BI, SAP, etc. This recommendation is a guaranteed success factor as people tend to stop thinking whenever they discuss the SOA panacea. Maybe, they'll recognize that none of this technologies has any relevance for the project, but then it will be too late, anyway. And if the project fails, there are lots of project members you could easily assign the scape goat role. Here, the "I am the Wizard of Oz surrounded by evil demons and incompetent engineers" argument will convince senior management. Take a financial reward from management for opening their eyes and then leave the company. Now, it will be a clear SEP (somebody else's problem).

Note, that knowing these recommendations does not imply that I have personally seen or applied any of them. Nor do I claim that the list above is complete. You may tell me about your personal experiences by adding a comment to this blog. I am open to any feedback :-)