Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal

Thursday, October 26, 2006

Live from OOPSLA 2006 (continued)

I will be less verbose in this second part of my travel report as I am summing up almost the whole OOPSLA from my personal viewpoint (it is impossible for a single human person to attend all those interesting things in such a large conference). Thus, my impressions are a little bit biased as they only reflect my personal research interests. And I am a lazy writer :-) So don't expect me to give you every detail. You should definitely attend OOPSLA yourself next time.

I met a lot of various friends and people during the conference days (that is, as some say, the most important topic here). And as Terri Parr (conference chair) proposed I met a lot of new people. That was a really exciting experience. One of the funny stories is that I met a professor and it turned out that he was Axel Schreiner, one of the famous german computer scientists who left Germany and now lives in the US (I read his books on compilers and many other issues years ago during my university education).

This is the official start of the main conference. It is the 20th OOPLSA aniversary and it is back there where it started: in Portland, Oregon. This year, OOPSLA is collocated with Dynamic Languages Symposium, GPCE, and PLoP. Terri Parr who is the conference chair announced that there are 1140 attendees, 430 from overseas with 460 first timers. Oregon locals: only 59! This proves that OOPSLA is a true international event.
The program comittee accepted 26 out of 157 submissions with 5-7 reviewers per paper.
2 essays out of 9 were accepted. In the Onward! track 10 submissions of 22 were accepted and 2 will appear in the proceedings. Terri told the audience in her greeting talk that Siteseer rates OOPSLA as one of the 50 most influential conferences.
The first keynote (Onward!) by Brenda Laurel (California College of The Arts) was on designed animism. Basically, it was a little bit esoteric but nonetheless interesting and inspiring. Brenda illustrated that every animal and plant has a "soal". She mapped all those ideas of animism to distributed and pervasive systems. And the basic driver for all of this is having fun. That's a rather short and incomplete explanation, I know. But thinking about it is left to you as an exercise :-)

In the language track (Research Papers) I heard a talk on how to dynamically extend typs by expanders. Instead of inheritance, source code change it is better to use adapters. With tool support this is a powerful feature.

Linda Northhrop (SEI) gave an invited talk on ULS: (Ultra Large Scale System). The bottom line was "Scale changes everything" in terms of software development. Take the Internet as an example with a decentralized, large number of users and data. Other applications include Healthcare Infrastructure, Homeland Security, Networked Automobiles, Military Systems (platforms, sensors, weapons, warfighters). ULS is a research study sponsered by DARPA. Goal: US Information Dominance.

Problem of such systems include continuous changes and sustainability. These systems are much more complex than what we have seen before. Heterogeneity is an additional issue. From the technical challenges, the US Army is concerned that complexity could be beyond reach (what about 1 billion line systems). That's why they initiated the study by the SEI Team and an Expert Panel. Process: Meet with experts in different fields at a 3 day meeting. Result was - as you might have expected from an expert group - a Research Agenda. They took a reductionism approach according to Linda.
First question they asked themselves: when do we encounter an ULS? Possible properties to look at could be:

  • size: in lines of code
  • number of connections
  • perception
  • number of processes, interactions
  • number of overlapping policy domains
  • number of people involved

Expect unprecedented scale of some of these properties in an ULS. ULS will be interdependent webs of systems, people, and other things. These are basically webs of systems each of them revealing internet scale (!!!).

Characteristics of an ULS system:

  • decentralized
  • inherently conficting, diverse, unknown requirements
  • continuous evolution and deployment
  • heterogeneous inconsistent changing elements
  • erosion of the people/system boundary
  • failures are normal
  • new paradigms of acquisitions and policy

Today's approach for large systems is to engineer top-down, controlled and use an agile perspective for small projects. For ULS a new perspective is required.

Metaphors to make that more obvious: today we build buildings, ULS is like building cities. Thus, guidance/rules are required. Another metaphor could be to think in ecosystems (independent systems, interactions, ...). ULS can be considered a socio-technical ecosystem with a lot of competition for resources and varying policies settings. In ULS we deal with ecosystems made of people, software, hardware, governments. Hence, engineering cannot be the metaphor. There are many conflicts with our approaches today. ULS == decentralized which is in conflict with today's centralized approach. Requirements in ULS are mostly unknown while today we believe that requirements can be known in advance and they do only change slowly. One of the additional tradeoffs will be stable continuous evolution: today we think of system improvements in intervals. In an ULS we face huge heterogeneity while in today's software systems we believe that effect can be predicted, configuration is accurate and in control. Social interaction is completely not considered today.

Challenges in three areas: design and evolution, orchestration and control, monitoring and assessment.

To investigate in ULS we need to establish interdisciplinary teams, learn from existing projects. Game theory might be a good point as well as the consideration of other existing networks (biology, body, ...) .

The ULS team came up with 7 areas of research

  • Human Interaction: "understanding users and their contexts"
  • Computational Emergence: digital evolution, metaheuristics in software engineering, algorithmic approaches in software engineering
  • Design: on all levels: software, rules, regulations, .... (the whole ecosystem)
  • Computational engineering: more expressive languages, other kinds of modularity, formal techniques
  • Adaptive System Infrastructure: decentralized configuration and evolution
  • Adaptable and predictable system quality: qualities - what do they mean in an ULS context
  • Policy, Acquisition, Management

Find the study as pdf in http://www.sei.cmu.edu/uls. A collaborative network is beginning to develop.

After the keynote people geve the feedback there was a little too much US military proximity, but that the topic is highly relevant.

There has been a Panel on the same issue with people such as Doug Schmidt, Gregor Kiczales, Linda Northrop. Here are only some excerpts from the panel to give you an impression.

  • Gregor, Doug: First we need to solve problems of small systems before moving to the ULS.
  • Kevin: Software as a problem must be solved in general.
  • Martin who is focusing on monitoring and repairing systems: uncoordinated growth require repair and monitoring support. Resources must be available to enable growth.
  • Ricardo: Let us play with such systems to find out before actually deploy them.
  • Peter: Lot of lessons we have not learned.
  • Neil Harrison (not a participant of the panel) cited Conway who said that organisations influence ULS. Look at your organization and that's how your systems will look like.
  • Peter: Deficiencies show lack of communication in teams. Conway's law is fundamental.
  • Doug: ULS is about heterogeneity. Can be challenging when parties involved are not really in favor of cooperating. Important: Building cities instead of buildings. Design process is important. Architecture consists more of rules. We need more abstractions.
  • Gregor: Ambiguous Meaning. Now we have not even a common understanding of most terminology.
  • An attendee from the audience asked about precursors of ULS. Doug thinks, current systems like military systems are precursors. A system that integrates all critical government systems could be a precursor. Linda mentioned, Internet+Wireless (devices) combined could be a precursor.
  • Another attendee asked about metaphors the team came up with? Cities, Ecosystem, Human interaction were the ones. The panel was a little bit inert and difficult as most participants are more technology biased. Sociology has many good points as Doug mentioned.

At Tuesday evening there was the most emotional part of OOSPLA. IBM invited to a dessert buffet and a memorial to the life of John Vlissided who died so early in late 2005. Desserts were John's favourites if you may wonder. Interestingly, the last time I had personally met him on the last day of OOPSLA 2004 in Vancouver we had talked about - cookies. If you don't know John: he was one of the Gang of Four authors of the seminal book on design patterns. Erich Gamma, Ralph Johnson, and Richard Helm (the other authors) talked in the evening event about their memories working on the book project with John. They also showed some video excerpts which really revealed to the audience what smart and nice guy John has been. My personal recommendation. Search for John Vlissides in Google and learn more about him.

A Keynote on A Growable Language given by Guy L. Steele introduced Fortress. See http://research.sun.com/projects/plrg/ for more details. Fortress design goal was: to do for Fortran what Java did for C. Guy who was involved in Java design mentioned, there were many proposals to add scientific computing features which were rejected to keep the language small and portable. Thus, funded by US DARPA IPTO (High Productivity Computing Systems Program), Guy and his team started a new language design from scratch. Goal: availability by 2010 and beyond. Similar work is done by IBM (e.g. X10). Fortress offers accelerated development even for desktops (Multicore). Key idea of Fortress: - don't build a language - grow it. E.g., many people would like to add a lot of different primitive types. But no way! That's not the way Fortress will work. Whenever possible, Fortress' approach is to integrate a library instead. Hence, library designers need control over syntax and semantics (not just over method calls). Fortress offers a few primitives: binary words of different sizes, linear sequences, heap sequences, user defined parametrized types, user defined polymorphic operators. The compiler leverages aggressive type reference aggressive static and dynamic optimization. Libraries provide types such as lists, sets, matrices integers, floats ... with physical units. Data structures might be local or distributed. Fortress also intends to ease parallelism. In addition, it contains a programming notation that is much closer to Math. Instead of having to deal with Unicode characters, Fortress offers an ASCII (Wiki-like) notation. Example: A UNION {1,2,3,4} .
The strategy of the language specification team consists of studying existing applications, studying how libraries can improve coding, and adding language features. Replaceable components in the langauge system avoid a monolithic standard library and encourage change. Fortress tries to make abstraction efficient, The type system basically consists of objects and traits (like interfaces, may contain code). Multiple inheritance of code not fields is supported as well as multiple inheritance of contracts and tests (automated unit testing). Traits and methods may be parametrized. Primitive types are "first- class data and control models". To implement Fortress more flexibility is required than just compiler optimization. Things such as transactional memory (atomic blocks, ...) will be relevant. Libraries may define which operators have infix, prefix precedence, whether juxtaposition is meaningful, what operators mean.
Parallelism in Fortress was not a goal but a pragmatic compromise. Parallel Programming is still difficult and error-prone. The question the Fortress designers asked themselves: can we encapsulate it in libraries? They did it to some extent but also added some implementation. Loops are parallel by default in Fortress (library convention) .

Charles Simonyi presented Intentional Software Corp tool in Onward! that allows to combine different DSLs and use an IDE to have different perspectives on the same source code. Very interesting and inspiring talk. The tool is currently evaluated by Cap Gemini and ThoughtWorks. Basically it allows to you view your code as code and also have some other perspectives at the same time. For example, if you implement a state machine, state diagrams could be made visible in your IDE editor. This is an excellent idea that can not be described in a textual blog. Expect more from this cool company in the future.

Another invited talk was by Joshua Bloch (Google, formerly Sun) whom many of you may know as the author of the Effective Java book. His topic was about how to design a good API and why it matters?

APIs are important assets for companies but might also be a liability. The Process of API Design basically consists of

  • Gather Requirements with a healthy degree of skepticism
  • Start with a one page document
  • Write to your API early and often
  • Writing to SPI even more important
  • Maintain realistic expectations.

General Principles that are useful in this context:

  • APIs should do one thing and do it well (also important here: use comprehensive names)
  • API should be small as possible but no smaller.
  • Implementation shouldn't impact API. Minimize accessibility of everything.
  • Names matter - API is a (little) language.
  • Documentation matters- document religiously.
  • Consider performance consequences of API design decisions.
  • API hast to co-exist peacefully with the platform.

In terms of class design Joshua gave the following advice:

  • minimize mutability
  • subclass only when it makes sense
  • design and document for inheritance or else prohibit it

Method design is another point he mentioned:

  • don't make the client do nothing the module could do (otherwise lot of boilerplate code is the result)
  • don't violate the principle of least astonishment
  • fail fast - report errors as soon as possible after they occur
  • provide programmatic access to all data available in string form (e.g. stacktrace as one string vs. everything as set of elements)
  • overload with care
  • use appropriate parameter and return types
  • use consistent parameter ordering across methods
  • avoid long parameter lists
  • avoid return values that demand exceptional processing

Exception design is another important matter:

  • throw ecxceptions to indicate exceptional conditions
  • favor unchecked exceptions
  • capture information in exceptions.

On thursday I gave my two tutorials (Software Architecture - Strategies, Qualities, Principles as well as SOA from an Architectural Viewpoint). Both were well crowded and I got a lot of positive personal feedback after the tutorials. Neil Harrison and Scott Meyers both attended my Software Architecture tutorial and asked smart questions (as I had expected it). I will put PDFs of these tutorials on my Web site. Thus, I don't go into the details here. The most interesting event of the day was a fire alarm during my first tutorial. There was a power outeage in the part of Portland where the venue was located. Hence, I made a reading session out of my tutorial while the beamer didn't work. Guess, that's at least something the attendees will remember :-;

The next day I left back to Munich. I am really looking forward to the next year where OOPSLA will be located in Montreal.

Tuesday, October 24, 2006

Live from OOPSLA 2006

This is the third day of my OOPLSA/GPCE 2006 experience. This year, OOPSLA takes place in the Oregon Convention Center in Portland.
I spent half of the time talking to other people such as Rob van den Berg, Arno Zimmermann, Eric Mejier, Jimmy Nilsson, Neil Harrison, Peter Sommerlad, Doug Schmidt, Arno Haase, Markus Völter. Conferences are an ideal place for meeting other interesting people.
Yesterday, I attended a tutorial by Arno Haase and Markus Völter on Domain Specific Languages and Model-Based Development. As tool chain they used OpenArchitectureWare and Eclipse. As an running example Arno and Markus illustrated the domain of state machines. The tutorial was absolutely entertaining. Now, I know about all of the important buzzwords. Just kidding :-) Indeed, the main benefit of the tutorial was its pragmatic approach. Funny to see Markus torturing Eclipse to get the most out of it.
Today, I planned to see Niclas Nilsson in his tutorial how to write code generators. Unfortunately, the tutorial was already crowded with people. Tutorial speakers like me can attend other tutorials free of charge if there are some seats left. And that can be a challenge. As an alternative, I intended to participate in a tutorial on programming the Sun SPOTS robots. Guess, what? Thus, I eventually attended a tutorial by Jeff Garland and Richard Anthony, both experienced practitioners for building large scale systems. They talked about building solid distributed enterprise software architectures. In details, they focused how UML and architectural principles can help for this purpose. It was the intent of the tutorial to illustrate how to effectively use UML and architectural principles to design and document an architecture. I missed a little bit the architecture principles aspect. All in all, I can recommend the tutorial.
This evening, I will meet Eric Evans and Jimmy Nilsson for dinner. Before, there will be the Welcome Reception, the first event in a series of social events.

Wednesday, October 18, 2006

Active Web - formerly was Web 2.0

I am track chair of a track on Web 2.0 for the upcoming OOP conference (http://www.oopconference.com). In this context, I also agreed to give an introduction talk on what Web 2.0 is all about. Simple task, I thought in the beginning. Well, Web 2.0 is about social networking and AJAX and tag clouds and YouTube and Digg and podcasts, and ... , you name it. The name Web 2.0 is really confusing, isn't it. But, what does Web 2.0 really mean?
From my viewpoint, there is a big commonality among all those concepts and sites. Everything in Web 2.0 is about (inter)activity.
In AJAX and similar technologies - the web page is not passive anymore. Instead, it contains code fragments that actively pull/push information from/to backend servers to achieve a better user experience.
In social networks users are not passive consumers anymore, but actively connect with each other. To enable this, web sites must actively connect different people with each other.
In pages such as Flickr, YouTube, Digg, Amazon people actively share information and media. Podcasts and VideoPodcasts allow people to become active content providers. Same for P2P networks, Wikis, Blogs, ....
In other words, Web 2.0 should be better renamed to reflect the aforementioned observations. It is about the evolution of the Web from a passive medium where we find a small group of active content providers and a large group of passive content consumers to a new Web where the boundaries between content providers and content consumers more or less disappear. This new Web provides technologies and means for Web users to switch from a passive consumer to an active provider almost immediately. It is a Web where everyone can participate. It is no longer a technology platform dedicated to large companies for marketing purposes or establishing Web-shops. It is more about social interactions and active participation. It is an open medium for everyone.
My suggestion thus is: let us rename Web 2.0 to Active Web. I think, this name much better reflects what the new Web evolution is all about.