Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal

Sunday, October 28, 2007

OOPLSA 2007 (Additional Info)

All keynotes are now available as Podcasts. Interested? Click here!

Thursday, October 25, 2007

OOPSLA 2007 [unplugged]

oopsla

I will just provide you with my unedited script on the OOPSLA 2007 which took place 21.10.-25.10.2007 in Montreal, Canada. I decided not to edit or beautify it. Hope, you are not offended by this fact. Yes, I know, this is a huge posting. Of course, the most important part is not included here: personal communication. This always has been one of the most important experiences on a conference like this.

Nonetheless, I hope you find it helpful to get an impression.

Some General Statistics:

Attendees 1225
90% male
60% newcomers
Most are architects, researchers, developers, also some testers
78% academy, 22% industry
Most 25-34, then in the age of 35-44
31% from rest of the world, the others from Canada and US

Conference Chair: Richard P. Gabriel (IBM Distinguished Engineer)

Tutorials I gave:
One on High Quality Software Architecture which was very well received. In this tutorial I explained different perspectives relevant to get high quality such as process principles, quality attributes, and architectural principles.

In my 2nd tutorial I addressed Software Architecture Refactoring. This is a new topic I already explained in other postings which is why I don't get into the details here. The tutorial was number one in terms of attendance. I got interviewed by infoQ (video) and will post where and when it will be available.

The 3rd tutorial was actually given by my team member Jörg Bartholdt on WS-* standards.

Tutorials I attended:
The one by Martin Odersky on SCALA (Scalable Language) which runs on the Java VM was an eye-opener. This language combines OO and functional programming. A free compiler is available for download. Highly recommendable.

I also went to a tutorial on product line engineering by Charles W. Krueger (BigLever). Very good introduction. Nonetheless, I found it a little bit disappointing as it focused on a concrete product, namely GEARS provided by the speaker's company.

First Keynote, Peter Turchi (Professor Literature/Poetry), Warren Wilson College
Some extract: Are we done? Can we write anything new? This is the typical pattern of repetition we experience in daily life.

Getting lost is important for imagination but is hard and requires some preparation. This especially holds for reading.

Exploration is part of the writing activity. Important is to discover things others haven't (examples: Galileo, Herschel, ...). "Seeing is an art which must be learnt". Example: upside down map where Australia is on the top. Different kind of seeing the world. How we see depends (in part) of what we want to see. Example: maps always show the bias of their maker.

"Being disoriented can be fun (and instructive)". "Perhaps being lost, one should get loster". Models/perspectives on reality.

Panel: Celebrating 40 years of language evolution: Simula67 to the Present and beyond
Steven Fraser (Cisco), moderator
J. Gosling: Played around with Simula, was an eye-opener, had a nice threading facility (in fact co routines), was driven by simulation, big question in next 10 years will be multi cores. Programming will be different. Other kinds of problems, 3D rendering for video games.
A. Hejlsberg: influences from functional programming, dynamic, declarative, Meta programming, lot of interesting things happening, e.g., LINQ, we need to think about newer models. For example, how can we deal with concurrent programming? No good solutions currently available
Ole Lehrmann Madsen, Aarhus: was a Simula 67 programmer, started BETA,  Simula 1 in early 60ies. Originally for Operation Research, then more general purpose. Extension of Algol, also a tool for modeling, sth. we are missing today
B. Meyer, ETH, Eiffel Software: One single environment for everything.
Guy L. Steele: cites some quotations on languages
Q: to Anders, Concurrent and functional programming - programming as a problem statement? Anders: you mostly say the what and the how. Too much "how". Need moving up abstraction level. LINQ also applicable to parallel querying such as in PLINQ. A lot of things to do e.g. in Meta programming. Bertrand: Functional languages may make procedural languages more flexible. Don't think functional programming can become mainstream. We would like to embrace the angel (mathematics) but we can't ignore the beast (hardware). Concurrency is biggest topic, changing little by little. Look at SCOOP.

James: Functional will be important. Only small percentage of community can handle functional programming. Humans want to be procedural. Guy: Likely programming will become very functional but not completely. Functional programming simply ignores state. Important for distribution.
Q: Languages changes programmer. Best way to make more creative. In which direction do you want to change?
Bertrand: Kristen Nygaard. Programming as modeling. Programming is understanding. Think about your system throughout the lifecycle. Wealth of inheritance, e.g. multiple inheritance. Generics. Idea of semantics: describe abstract logical properties. Ole: modeling is important. Teaching a student a conceptual framework for students, not letting them depend just on the language. It is not about just adding fancy features to languages. Simula was not originating from programming.
Q: Kristen started to talk about the future, mentioned OO was good but not sufficient. Started COOL (project on teaching programming) and STAGE (web like multi paradigm language. Make actors and give them a script). How about this idea?
A: (Ole): New challenges are beyond Simula concepts. Global pervasive computing cannot be covered this way because you don't know about what's around. Time and location did not matter when Simula emerged. STAGE was intended to extend Simula. Bertrand: Kristen was very proactive.
Q: Programming is not very satisfying. It is more about finding the right API? Will this happen to concurrency?

James: Smartness is  not sufficient (look at Silicon Valley). Sophisticated software for embedded systems where you have to know it must be. Anders: 40 years of evolution have not made that difference. Still have to write standard statements such as i=i+1. This could and should change. Correct? Such as Avionics? There is no shortage of problems.
Q: Why don't eat OO language designers eat their own dog food? Still same old ACSII style programming?

Bertrand: no one cares about characters. Anders: Meta programming is important. Ruby is successful not due to typelessness but due to Meta programming facilities. If Meta language and language are the same, then productivity will increase. But "types are better". Nonetheless, dynamic languages are often precursors.
Q: One way of interpreting STAGE is enabling hierarchies of virtual machines. Linguistic machinery.
A: Ole: Why did not others discover Simula co routines which Ole thinks is the same considered in hierarchical machines. Bertrand: Co routines mechanisms is like reducing 3D to 2D.  Extremely elegant but not a solution for concurrency. Preconditions do not mean the same thing. Correctness can not be as strict anymore. Co routines do not help you there.

James: co routines was only a "hack".  But it encouraged a programming style with independent code entities.

Anders: part of the problem with concurrency is the huge mass of different ways for it. Language designers should look what is lacking in languages. Such as using lambda expressions. There is not one single model.

Guy: too strong commitment to stacks. Useful but enemy of concurrency.
Q: Historically languages came from computation. Now, the physics must be recognized such as in concurrency?

Bertrand: task of languages is too abstract. Otherwise, we would have to read hardware-manuals to understand FORTRAN.
Q: More domain specific kind of languages?
A: Ole: Simula started as simulation language. Then added class simulation framework. Then added syntax elements for this framework.  Had to develop domain specific language. Anders: DSLs very quickly need elements from normal programming languages (type systems/expressions). They don't raise the abstraction this way. Instead of DSLs raise abstraction by general purpose languages.

James: involved in real-time programming. Lot of languages expressing QoS. Rest is just crap. Then people don't use the DSLs anymore, because rest of infrastructure never gets sufficient. Bertrand: modeling is not modeling the world. Relationship between both is very indirect. Whole idea of OO is to implement DSLs (specialized tools, libraries, syntax not required).
Q: Real world is concurrent. What are your favorite metaphors for concurrency and collaboration?
Anders: can not give a good answer. Typically not much  concurrency visible to the developer, even though underneath a lot of stuff involved.  Same for agent systems. The more hiding, the more successful.

Bertrand: correctness not the same on multiple CPUs. Testing feasible but not useful anymore.
Q: Multiprocessor is hype today. But what about solid state memory?

Anders: Solid state memory may have more impact in the next time. Hard to reason about how long time cpu cycles takes. No exact answer. James:databases? Just use RAM. Just use transaction lock.

Keynote on Second Life, Jim Purbrick, Mark Lentczner, Linden Lab
People can do anything, so they do anything to experiment. Demographics match demographics of real world. Lot of code running, teaching ordinary people to build software. 15% of 2nd life population codes. LSL script. VM - very low number of features. Must fit in 16k. Everything in message passing. 14500 regions each of them running in a separate CPU. Each of them running 1k-2k scripts. Massively concurrent! Scripts can move. Thus they must be able to be persisted.

Keynote by Frederick Brooks (The Mythical Man Month)
In history a lot of artists could provide their design alone (art, engineering, craftsmanship). This is not possible any more because of the increased sophistication of every aspect of engineering. We also get a hurry to get to market (early birds win). We need to have specialized expertise. More work: task partitioning, interface definition, shared element standardization, common style definition, integration and testing, on-going interface interpretation & reconciliation. Challenge is conceptual integrity. Many great engineering designs are still today principally the work of one mind, or two (Cray, Menn's bridges, a set of beloved software systems). Fan Clubs: FORTRAN, VM/360, UNIX, Linux Pascal, C, Macintosh, APL.

No Fan-Club: COBOL, OS/360, Windows, Algol, PL/I, PC, Ada. Differences between both: Fan Club means: developed by one mind. No Fan Club: design committees.
How to get conceptual integrity? Design as interdisciplinary negotiation? NO! Mill's Chief Programmer concept - a supported designer. A system Architect for design beyond one chief designer. The architect: agent, approver, advocate for the user.
A system architect who really cares. One user-interface designer. Documented assumptions: user population characteristics, application, and its future use and development, BETTER TO BE WRONG THAN SILENT OR VAGUE!, forces discussion, enables sensitivity analysis, directs verification efforts.
A Style Sheet: consistent styling of details is the hallmark of conceptual integrity.
The Cathedral and the Bazaar (Raymond's brilliant essay on Linux). Bazaar as an evolutionary model. No committee design - each piece has conceptual integrity. Emphasizes early and large scale testing. Marshals many minds for fixing, not just testing. The market votes among alternatives by adaptation.  based on a gift <-> prestige culture. Among people who are fed anyway. Works when the builders are the clients (know requirements from personal experience). Also applicable for an air traffic control system? Hopefully, not!
When does collaboration help?
- determining needs from users (more users => more diverse questions)
- conceptual exploration - radical alternatives
- not conceptual design nor detailed design (distinguish sharing design from delegating design) But pair programming wins.
Design Reviews
- especially with different expertise
- need and exploit richer graphical representations
Some Collaboration Caveats
- Real design is more complex (than in textbook examples)
- demands change control
- Collaboration is no substitute for
  the dreariness of labor and the loneliness of thought
Telecollaboration
Why?
- specialized skills
- geographic dwelling presence (national+cultural, city vs. suburban vs. rural)
- work around the clock
- cheap labor
- political factors
Example: Airbus 380. Telecollaboration plus resident ambassadors, a plane between Bristol and Toulouse every day.
Making telecollaboration work
Face time is crucial, low tech often suffices, shared document important then voice then way behind: videoconference (vital issues, people insecure, interviewing strangers, organizational or national cultures different).
Clean Interfaces: make big differences in error rates, in joy of work. Telecollaboration study: Mostly technology driven, not design-driven. A library shelf approach: 19 of 20 book on tools.

Panel "No Silver Bullet" Reloaded - A retrospective on "essence and accidents of software engineering"
Steven Fraser (panel), Frederick P. Brooks (University of North Carolina), Martin Fowler (ThoughtWorks), Ricardo Lopez (Qualcomm), Aki Namioka (Cisco), Linda Northrop (SEI), David Lorge Parnas (University of Limerick), Dave Thomas (Bedarra Labs)
Introductory Remarks:
Brooks: Difficulties in engineering can be separated in essence (structure) and accidents. Forecasted that productivity can only be achieved by conceptual work, not by eliminating the accidents. Bold statement: no technology will appear in ten years that gives a tenfold improvement (measured from 1986). Did not happen.
Parnas: There isn't a silver bullet. Silver bullet: no skill needed to use. Interesting question: why did Fred feel the need to write the paper and why do we still ask the question. Two points: designing software is hard, cannot cope with simple  approach. Point two: poor work man blames his tools. Developers are poor men. Looking for some magic. Myth we have had such a lot of progress. But that isn't the case. Why don't we use the lead bullets?
Linda: read the paper when everyone talked about the software crisis. Used the points in the paper over and over again. Figuring out what to say not how to say is the problem. Some progress has been made. Product line engineering: ten fold increase but that are only exceptions. Modeling is the essential point. People tend to focus on languages instead. We need great designers and cultivate atmosphere of hard work, also in interdisciplinary areas.
Aki: Is responsible for Cisco regarding services. Discipline has extended tremendously. Tools help even non-software developers build functionality. Software engineering takes a village to develop a software system. Many issues to cope with. It is a team effort.
Dave Thomas: Considers Fred's article as a challenge. Current state of OO technology is a disaster. No hope. Very hard to build stable products on top. Things may be created by smart people but the reality is normal people are challenged. Stuff too complicated. It is not sufficient to give certificates. Need more competence. Object are great but they have gone amok. Same for agility. Successes in niches.
Ricardo Lopez: We got a lot of silver bullets but they are killing us. Silver bullets comes from fearing. Trying to avoid what we fear. Fearing complexity is like fearing life. But this is elegant. Search for productivity is another driver (we are always trying to optimize). Silver bullet: we need to address and embrace complexity without fear. You are the silver bullet. Collaboration helps sharing experience which gives an order of magnitude. Silver bullet is inside of us.
Martin Fowler: distinction between essential and accidental has had a great influence. [Changes to a werewolf with cries of pain and will be the bad "person" in the panel arguing against the rest of the panelists]. OO is a dangerous and evil idea, but I could overcome it. Great ideas but no one actually does it. I love multicore concurrency systems. Use of prebuilt structures is a great theory. Only helps if libraries are good. Also requires good designers. No one outside OOPSLA does fortunately understand this. Invisibility of software development helps even more, at least to ordinary population. Rapid prototyping. One of the surprises ho many waterfalls I saw. Even lead bullets help, because people are crazy  about silver bullets.

Questions by the audience
Q: Paper was too successful. Often used by managers to reject new techniques.
A: Fred: If technologies are not addressing the sense they are not addressing the right thing.
   Linda: More address the customers and the business case and product needs but don't sell the technology. Other reason for silver bullets is greed
Q: How does this apply to individual productivity?
A: Agility is the way to let people grow. Make best people role models for not so good developers. Establish opportunity for others to learn.
Dave Parnas: Amount of training time matters. If small learning curve, then it is a silver bullet. Otherwise it is a lead bullet. If it is a silver bullet, sell it to IBM. Otherwise educate.
Dave Thomas: If it takes that small time, it is a scam. Find the better people and get them together.
Martin [aka werewolf]: There are not too many good people. Even if, people don't try hard enough to collaborate. People want everything easy and comfortable.
Q: What are about smaller lightweight teams with single individuals?
A: Dave Thomas: comes back to requiring good developers which is rarely possible. Leadership helps.
Fred: Growing teams. Book peopleware is extremely good. The Carolina Way: book how to make a good team out of Madonnas.
Linda: it takes a leader. Management is not enough. We need to be disciplined.
Martin: Peopleware is a dangerous book. Fortunately it is hard work to manage a team. It is more than Microsoft Project. Very easy to derail.
Q: How do we measure productivity?
A: Martin: you can't measure your productivity? You can't argue excessively about objects versus functions.
Ricardo: Metrics are crap. You can't measure the aggregate productivity. You may could derive micro economics from macro economics.
David Parnas: Preferred to have non-quantitative measures. E.g., give it a stranger and measure how long it will takes. That’s what we would quantify.
Aki: Often people only want to increase to get more. Then she asks what she/he means by productivity.
Q: What about non-linear increase of productivity?
A: Aki: often complexity is increasing by qualities such as security, transactions, ....
   Dave Parnas: critisizes web sites. web sites are often done by smart people which then no one is able to extend
   Linda: We need to have more design upfront.
   Dave Thomas: Communication is often the issue. Get rid of all the middleware and business objects crap.
Q: Mostly people address in environments on accidents and incidents. When will we have an environment that handles essence?
   Aki: Tools have improved.
   Dave Parnas: When going to building. First build the foundation then address the rest.
   Martin: Communication between users and developers. Communication here is hard. Business people don't want to talk to developers. Developers have no social skills. Software people get frustrated so that they loose interest in business case. Then they start building large infrastructures to overcome boredom.
Q: By Brian Foote. World runs on bad code. Software is a success story. Shouldn't we teach people how to write more bad code?
A: Ricardo: Wonderful success (civilization is embracing software success) makes us more vulnerable. Code should contain some self-correction. No technology can be perfect. Focus on guaranteeing code won't cause big problems.
David Parnas: Excellent developers writing code no one else can cause a problem.
Dave Thomas: Mismatch between smart minds also might be a problem.
Q: We have unrealistic objectives. Silver bullet could be training bringing IT people to business people?
A: David Parnas: might be good. But the same would be to have every driver being a mechanics. But we need people who make things for other people.
Q: Articles by Brooks and Parnas were silver bullets. Only we need no silver bullets now, then thus this mean non one can come up with one?
A: Dave Parnas: I did not provide a silver bullet. Nobody is against good ideas.
Q: It took a thousand years to go from geometry to calculus. How do we finish objects? How do we provide good libraries?
A: David Parnas. Calculus is a wonderful thing. It is a lead bullet but no silver bullet.
   Fred: better when system satisfies somebody than anybody.
   Ricardo: does not consider metaphor valid (romans just ignored Archimedes).
   Dave Thomas: best way to get components is to stop people developing frameworks. Most frameworks can only be used by experts. Better provide functionality as component. Is not optimistic that this is approaching.
Final statements
Martin: software development has made progress. But werewolf is not harmed. People underestimate. Human beings have this optimism.
Ricardo: We're in trouble. Werewolf is annihilated, but then new werewolves arrive.  Synergies with your peers are important.
Dave Thomas: New generation can face the challenge.
Aki: Search for creating silver bullets, we have made it easier to develop. But highly sophisticated developers are required. Complex systems are still there but we will have to appreciate there also system that does not need that skills.
Linda: We are continue to try getting better. Focus in essence not on accidents.
David Parnas: It is unfair to criticize waterfall model. Because some notions have just been misinterpreted. We are making progress by building simpler systems, not by more complex systems. My dog does not fear werewolves because he does not try to get simple answers.
Fred: No field of engineering where people look more thoroughly on other's work. Very dangerous thing.

Keynote: ELEPHANT 2000 for the year 2005: A Programming Language Based on Speech Acts, John McCarthy (AI, Lisp, ...)
Natural language offers semantic features that are absent in present programming languages. Example: Passenger has made a reservation. Nothing said about databases here. Name cause of slogan: an elephant never forgets. Reservation is made to a virtual object. Referring to the future ("baggage handlers ready one hour before flight arrives"). References to the past by suitable data structures. Compiler has to invent them. References to future are more difficult (not always possible). E.g.: Prediction to actual arrival difficult. AI maybe required.
Speech Acts: initiated by "ordinary language philosophers" who didn't like logic. They study sentences without truth values. "I now pronounce you man and wife". Not imply an assertion.  "I sentence you to be hanged by neck until dead".  Speech acts include offers, acceptances, statements, questions, command, ... One can also state, describe, assert, warn,... Personal intent to be considered (why does someone do/say sth.).
Programs that buy and sell good and services make commitments and receive them.
They undertake financial obligations on behalf of their owners.
Correct performance includes fulfilling obligations and insisting these to be fulfilled. They are operating in society.
At some execution point a program may execute a statement asserting the intention that variable x will remain less than y.
Is the execution correct so far? Will this process terminate? Internal speech acts give a form of intrinsic correctness. For example, correct programs carry out their intentions.
Two kinds of external specification: Input-output specifications depend on program and language. Verification can be done using these.
The accomplishment specifications. Depends on facts about the world. "Customers will be satisfied with a service".
Elephant programs have both.
Arguments against verification because people have confused these two kinds.
Speech acts philosophers  distinguish illocutionary and perlocutionary speech acts. "He told her" vs. "he convinced her". To some extent, this corresponds to input-output and accomplishment specs.
Look at paper!
http://www-formal.stanford.edu/jmc/elephant.html

Then he explained the history of Lisp. How they came up with AI (Minsky and him, how the Lambda calculus was partially used, how garbage collection was invented)

Keynote: David Parnas - Precise Software Documentation - Making OO work
What is OO - A Buzzword for Our Times. My meaning: "design software by data holding objects making the job easier", no special language necessary, languages supposed to make it easier, ...
separation of concerns
information hiding modules (invented by Parnas)
- stresses work assignments, flexibility, substitutability
abstract data types
...
No conflict!
It is a recipe for disaster

Same examples used over and over again
Reacted to the "hiding" alone
you have to tell them something - accurately, precisely
read the 2nd edition of The Mythical Man Month
There was an earlier article on the interface documentation
For 30 years, I have known that documentation was the other side of the coinlevels of abstraction (relation never clearly defined, stress on subset ability - many different ideas)

35 years later
never doubted the truth of the information hiding theorem information hiding is no empirical result. It is a theoretical result. Think of subsystems A and B with fixed interface.

Requires empirical verification.
- that people can know what to hide
- that the interface can be efficient
- that we can describe interface without giving away secrets

Old Problem: Documentation
Apeldoorn, 1969. Philips. Were unable to write specs for software components that were complete and precise. Parnas found it very difficult. Components had to know a lot information about each other such as data structures.
Still a problem. Think of Microsoft. Inability to provide good professional reference documentation is costing us millions maintaining complex products.
Role of documents in engineering
record key design decisions.
binding on everyone and fully controlled.
precise documents that use mathematics.
are not introductions or tutorials.
are not extracted documents (javadoc)
show "separation of concerns"

40 years software crisis
obviously not a crisis
underlying causes

- lack of discipline, careful choice of interfaces and structural decisions
- lack of review

Decisions that are not fully documented are not taken.

What is documentation?
Practical tool (not just theoretical achievement)
repository of info
convenient reference (organized like a dictionary not like an introduction manual)
easier to use for information than code
structured to avoid inconsistency
quicker and more authorative than trial executions
useful before, during and after the coding

Computer Science  and Documentation
Managers and engineers bemoaned inability to document
not just fragments of software
programming in another language
regarding documentation just as an essay or commentary
if you take the term "engineer" seriously you must ignore those views.

Verification and Validation
documents state what we want to verify
what we get to work with
used to support testing
Supports "divide and conquer" inspection and verification.

Properties of documents
-accuracy
-precision
-consistency
-completeness
-ease of reference
First three can be reached through mathematics. The last ones by better notation and organization. All require content

definitions for each document.

Rules
Don't mix intro and reference parts
never rely on words
mathematics only way to be precise but
- expressions must be simple and easily parsed
- interpretation should be direct
Only relevant information
Everything on one place

Clearly Defined Roles
-description: facts about products.
-specification: only required properties
-full specification: states all requirements
Same notation may be used for all 3. Matter of intent not notation. No such thing as a specification language.

Documents just formal methods?
Goal is to organize information for easy retrieval.
no mathematical model needed. Proof is not the main goal.

Content Definitions - when and how?
Often organizations specify formats but not the content
we need content definition to
- know what to put where
- check for completeness
Each document describes some mathematical relations. Each document has a different range and domain.

Main Documents
- system and software requirements, system and software design, ...

Modules and Components
distinct but related concepts
module: tasks
same documentation method for modules, components, objects
- event descriptors  (description of pre/post values of visible variables)
- time as a variable if needed
- trace: sequence of event descriptors
- history: trace describing what actually happened
- document relations

Module=private data structure + set of externally invokable programs
design, usually in designer's head, should be written down

How to describe relations in a readable way?
Two ideas:
- Tabular expressions
- Relational Model
We need both ideas!
Triggered by practical experience.
[Mentions lot of different example projects]

Pre- and post conditions do not scale up.
Developed process for inspection.
- preparing specification what code should do
- decompose program into small parts  appropriate for the display approach
- produce the specifications and descriptions required for the "display approach"
- compare both specifications

Key is divide and conquer and precise documentation.

[PARTY TIME ON WEDNESDAY EVENING]

Conference Event: Cabaret theatre. Different food offered. A rock band, painters, an event where the strangest web page was selected.

[THE LAST DAY]

Context, Perspective and Programs [copied from conference program]
Gregor Kiczales (University of British Columbia)
Context plays a large role in our perspective on the world around us -- people see things differently depending on background, role, task at hand and many other variables. How do different contexts affect developer perspectives on software?

What different ways do developers want to see a program? What different ways do they want to work with a program? How does a program mean different things to different people? How does context influence perspective? How do different contexts and perspectives interact? Can these interactions be reified, controlled and parametrized? A broad range of work has explored these questions, but many issues remain open. We lack a general understanding of the concepts and mechanisms that can support the changes in perspective we need. We lack the ability to handle context and perspective systematically, easily and reliably throughout software development. Work is needed in a number of areas, from conceptual foundations to theory, languages, tools and methods. A truly satisfying handle on these issues may even require a material expansion of the foundations of computation -- or at the very least the foundations of programming languages.

Invited Talk: Intimate Information for Everyone's Everyday Tasks (Mark Bernstein, Eastgate Systems, guru on hypertext).

Remark: this was a very esoteric talk. To understand look at TinderBox (find the hyperlink below).

Speaks about conceptual integrity and different views. Talks about different computers and software. The better specified a project the closer it is to slave's work. There is more to life than regular structures. Lot of tools for dealing with regular structures. But what to do when structure is almost regular. Introduces cathedral/bazaar metaphor. Here is where Neo Victorian computing fits in. The ultimate aim of all creative activity is a program! (Gropius: "Das Endziel aller bildnerischen Tätigkeit ist der Bau!"). Computing is one of the seminal artistic pursuits of our time. Compares development of building with that of programming which went from unique (monolithic structures) to components and re-use. Philosophically, it is all about romanticism and realism. Writing hypertext is like writing. Writing is hard. Refers to teen's diaries. Things we want to remember or think about. Other universe: filling out forms. Lot of trouble. Figuring out how to fill them out. We do think to often writing is about forms. Knowledge representation for everyone's intimate information. People who are writing for themselves. Constructive hypertext (interlinked information which we do not (fully) understand). At the beginning of your PhD you don't know the structure. Inheritance for people who don't program. Look at programs such as Tinderbox. It just saves typing. Prototype inheritance, no class objects. It is fast enough. Composition and inheritance. Write now, we can formalize later. Premature commitment is an important issue here. Don't commit too early, otherwise you get lost. Example: Organizing files and folders on one's computer. Good idea but people don't like and don't do it. Hypertext can help here. Lively data: persistent search. Different notions of inheritance: from class, container, mixins. We might need all of these. Hypertext is a new birth of freedom. Has become ubiquitous. It is invisible and seems obvious. We need to think about referential integrity. Need to address context, implicit narrative approach. Hyperlinks help readers tell where to go.

Next ooPSLA conference: 19.-23.10.2008 in Nashville

Tuesday, October 16, 2007

S-E-P

You may have heard about Douglas Adams SeP principle in the famous Hitchhiker's Guide to the Galaxy. SeP stands for Somebody else's Problem.

The SEP principle represents an ubiquitous principle in software engineering. You definitely have heard about it or even experienced it personally in some projects.

One typical example I often hear relates to software architects complaining that their management agreed on the requirements of a concrete software project without asking them for feedback or help. The management then throws the project specification over the fence to the software engineers which is when the whole mess begins. Involved engineers are doomed to fail due to incomplete, inconsistent or even missing requirements. Yes, indeed, all these guys from management are morons like those illustrated in the Dilbert cartoon series.

But wait a minute! Is it the job of a software architect to take a specification of requirements  without any review and feedback? No, that is not the job description of an architect, at least not from my point of view. As soon as you are asked to design a new system, it is a must to consider the business case, the requirements, the risks and then to decide whether this possibly could end up as a success story or rather develop to a dead man walk. If you find any problems in a project context, it is your duty to inform the other stakeholders about your findings and ask them to solve those issues before starting the project. If they don't accept any changes and your gut feeling tells you this project is doomed to fail, the last resort would be rejecting to participate in the project. Sounds rather harsh but consider the other option: If you take responsibility as an architect in any project then you are signing a kind of virtual contract where you fully agree with the whole project specification including the project organization, the process, the requirements, etc. This means you cannot simply finger point to your management afterwards when the specification turned out to be questionable. You have become part of the system and thus it is your fault too. And it is definitely not a SEP.

What this means for software architects is the fact that architects are not just puppets on a string controlled by some evil guys in HR or Senior Management. They have responsibility and should feel empowered by management, because if they don't, the architect's role in their organization is either undefined or not well defined. Draw your own conclusions in this case.

So from now on: Whenever you feel inclined that a problem you face in a concrete project setting could be a SEP, think about it again.

Sunday, October 14, 2007

OOPSLA 2007

I am looking forward to attending this year's OOPSLA in Montreal. It is not only the excellent conference, but also the great people. And of course I am able to meet a lot of good friends. Some of them I only meet at conferences even though they live not far from me. Maybe, this is a kind of conference law.

My two tutorials on Architecture Quality and Architecture Refactoring got a remarkable number of attendees. I am used to speak in front of large audiences but I never stop being a little bit nervous. Let us see what feedback I will get.

But of course, I am also very happy to see the great work of a lot of other people. I am sure we will see some fascinating stuff on Software Architecture and some other cool topics :-) Friends like Doug Schmidt will also give tutorials which you should consider a must if you are interested in this architecture stuff. A member of my team, Jörg Bartholdt, will present on WS-standards. As he is a superb expert in this field, I am sure he will do a very good job.

If you are attending OOPSLA and reading this blog, please, don't hesitate to get in touch with me. I am really happy to meet other people (especially you) interested in the never ending story of architecture and engineering. I am just including a small picture so that you may recognize me (or are able to escape if you don't like what I am writing here, but I accept any feedback :-) 

micha_foto

Sunday, October 07, 2007

The Great SOA Swindle (Beware of hidden humor!]

You might have noticed it or at least you got a little bit suspicious. Something strange is currently going on in the IT universe. People who you highly respected a few years ago have been caught by a serious disease. Now, they behave and think in unprecedented ways. You will hear them constantly producing sentences that sound like "yada yada yada SOA yada Governance yada BPM yada yada EAI yada messaging". My code name for the disease is WSA (Web Service Addiction), sometimes also coined SOA (Serious Opinion Anomaly). Their personal background does not matter either. I know C++ nerds as well as enterprise Java developers that were both caught by the SOA trap. Even mice (MIcrosoft Centric Engineers) got brainwashed. How do you recognize a person that carries the SOA virus? It all starts with loose coupling. Obviously, these poor morons loose the coupling to their own brains. You will also detect a serious speaking disorder. If you listen carefully, they will constantly repeat words that sound like "WSDL", "WS" or "XML". Any relationship with the dark lord in the Harry Potter series might be completely coincidental. There is a kind of Turing test to prove whether another person is suffering from the SOA disease. Just, tell this person about a particular software engineering problem you are currently facing in your project. If your peer immediately (without further considerations or detail questions) tries to convince you that SOA is the appropriate solution, you're done.

Don't consider all this just a SOAP-opera. And don't wait until you REST in peace. Act now! This dangerous virus is spread and distributed on purpose. A secret organization called CIA (Computer Integration Architects) is responsible for all the mess. Companies like IBM, Microsoft, SAP, Google, Oracle, <name your software vendor of choice> want to make us believe in a service paradise, while their actual goal is to increase stakeholder value. I bet they once secretly met discussing what new problem they could come up with in order to offer new products (sometimes also erroneously known as solutions).  And that's how Web Services and SOA were born.

Interestingly, only a few Web service success stories actually exist. And among all of the SOA success stories, most do not use Web services at all but a kind of messaging infrastructure. Now, guess how old messaging middleware is. Yes, we already have known and used the SOA paradigm for decades.

With other words, we are currently facing a giant SOA swindle created by a secret network of various contributors such as companies, media, or fellow engineers. Their actual goal is to achieve world domination. But that is another story and I will address this issue when talking about the truth of Web 2.0.

Saturday, October 06, 2007

Dealing with Human Affairs and Political Correctness

Software architects often constitute a sort of communication hub, because they sit right in the center of software development projects. I don't want to claim they represent the most important role. What I mean is that software architects have to deal with all the other stakeholders, be it customers, requirements engineers, testers, developers, and other persons involved. As a lot of communication happens with software architects, they need to take responsibility for the strategic and tactical design. This empowerment also bears some risks. For example, architects may  be "leveraged" as proxies in some (political) battles. Stakeholders may try to gain influence to architects in order to achieve their own goals, no matter whether these goals are expressed in an open manner or part of a hidden agenda.

Let me give you some concrete examples: what if a product manager asks you to use a particular technology from vendor X, even if the requirements in your project demand a completely different solution. Or suppose, engineers from different locations prefer to follow their own local interests instead of contributing to the overall project objectives.

You may be surprised how often software architects have to deal with such issues. And, of course, this also holds for the other roles - architects are by no means better beings than developers, testers, etc. And I claim that political issues pop up in every development project with more than one person being involved. But what can we do about it?

In my experience, the only possibility is either to quit your job in case the political quarrels eat a lot of time and resources and there is no chance, no matter how hard you try, to stop these quarrels. I bet that many of the 70% software projects that allegedly fail, fail to a large extent due to such issues.

The other possibility is to establish an open team culture with all people involved abiding to the rules of openness. Some of the rules could be: (1) no hidden agendas, (2) project goals over all other goals, (3) the project manager is empowered to decide all project-relevant aspects, (4) all issues are addressed in an open manner with all involved persons, (5) every project member must be treated with respect.

Obviously, we could fill a whole book with such recommendations.  Interestingly, it does not matter what kind of engineering process a project is following. Even in a pure agile setting such issues appear.

Maybe, we too often ignore that engineers and other stakeholders are human beings with various experiences, competencies, social skills, personal intents, and opinions. Thus, there is large chance of conflicts in a project team. For example, in a recent project some people were appointed architects and some others key developers. A couple of key developers complained not being appointed architects because they felt like second-class citizens. It did not help to explain that architects and key developers are just different roles, none of these roles considered more valuable than the other. 

We often hear a lot about technologies, concepts and methods. In a project and in every professional career these aspects often are less significant than social and other non-technical skills. Architects should always keep this in mind.