tag:blogger.com,1999:blog-51190252024-03-08T01:45:06.037+01:00Hitchhiker's Guide to Software Architecture and Everything Else - by Michael StalIf you are a software engineer: DON'T PANIC!
This blog is my place to beam thoughts on the universe of Software Architecture right to your screen. On my infinite mission to boldly go where (almost) no one has gone before I will provide in-depth coverage of architectural topics, personal opinions, humor, philosophical discussions, interesting news and technology evaluations.
(c) Prof. Dr. Michael StalMichaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.comBlogger205125tag:blogger.com,1999:blog-5119025.post-90377970383546932402023-09-05T20:54:00.013+02:002023-09-06T11:00:57.851+02:00The Dark Side of Crowdfunding<p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">This post is not going to cover any software architecture topic. Instead I want to share some impressions and experiences with crowdfunding platforms such as Indiegogo or Kickstarter.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Let me start with a success story: Bambu Lab was completely unknown when the upcoming 3D printer company started their X1/X1C campaign via Kickstarter. They eventually gathered almost 55 million HK-$ from 5575 backers. In the following months Bambu Lab completed the X1/X1C product line and sent all the perks to the backers. This new CoreXY 3D printer turned out to be a revolutionary, award-winning and extremely successful product which soon was followed by other products like the P1P and the P1S. Needless to say that Bambu Lab has been a huge success story with a happy end <span class="Apple-converted-space"> </span>for the crowdfunding company, the campaign owner, and the backers. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">One of the benefits of crowdfunding can be summarized as: crowdfunding platforms connect innovative campaigners and enthusiastic backers. They enable start-up companies and well established companies to get funding for innovative products.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">In hindsight, not all campaigns work that well. In some cases, campaigners fail to provide a product, only create an under average product, run out of money, or turn out to be scams. Year by year millions of US-$ get lost this way. It is never foreseeable whether a project will succeed, as it is the case with joint ventures. Reasons for failure might be infeasibility of the innovation, budget overspending, huge project delays caused by unfortunate conditions such as Covid-19, underestimation of costs, or sharp increases of prices for necessary components.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">While project failure can never be avoided, scams can. A chinese campaign owner collected over one million US-$ in his Indiegogo campaign featuring the world‘s smallest Mini-PC, but did not create any of the promised perks. After a while there would be even no communication between campaign backers and the campaign owner. It seemed as if the owner just had disappeared from the surface. When backers asked Indiegogo for help, the crowdfunding company did not feel responsible. They just disabled any further contributions, put a „this campaign is currently under investigation“-label on the project web site, but did never provide any results of the so-called investigation nor a refund to betrayed customers.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><b>Lesson 1</b>: crowdfunding companies do not care (too much) about backers. They earn money by providing a platform for different parties, treat backers as venture capitalists who are supposed to bear all the risks themselves.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Indiegogo, Kickstarter basically act like betting offices for horse races with almost no transpareny about the horse owners (aka campaign owners). Every participant in such scenarios bears high risks with the betting company being the only exception. Obviously, the rules between customers and the crowdfunding platform are defined in such a way that the bank (aka betting office) will always win.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><b>Lesson 2</b>: if you are contributing to a crowdfunding campaign, make sure, you can live with project failure and with complete loss of your contributions.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Every backer should be aware of this reality. She/he may lose her/his whole contribution or get an overpaid or even useless perk. Sure, the majority of campaigns does eventually succeed. However, there is also a significant amount of campaigns that fail. I do not bother about project failure despite of huge efforts of campaign owners. This is a known and acceptable risk backers should keep in mind when contributing. But I bother about scam campaigns where owners just take the collected contributions and vanish.</span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><br /></span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><b>Lesson 3</b>: If you urgently need a specific type of product, don‘t contribute to a crowdfunding campaign, but buy it from well-established sources instead.</span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><br /></span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><b>Lesson 4</b>: Currently, no safety nets for backers exist. Neither is there any transparency or accountability with respect to campaign owners. A campaign resembles a game or a bet on the future without sufficient transparency regarding campaign owners. </span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><br /></span></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"><b>Lesson 5</b>: Do not believe in videos and documents provided by campaigners. Consider this information as a pure marketing and advertising campaign. Never trust any promises, in particular not those that seem to be unrealistic or very, very challenging to fulfil. Phrases like „the world‘s first“, „the world’s fastest“ or „the world’s smallest“ should make backers sceptical.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">What could be done to avoid such situations? Or is the crowdfunding platform inherently unable to protect backers?</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">In fact, there should be a kind of trust relationship between all players in the game - yes, it is a game! To achieve the right level of trust, a crowdfunding company shall offer the following services:</span></p><ul class="ul1" style="-webkit-text-size-adjust: auto; list-style-type: "— ";"><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Personal identification</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"> of all campaign owners with official and legal documents such as passports, driver licenses, locations of residence. This enables companies like Indiegogo or Kickstarter to keep in touch with campaign owners and track them down. Sure, passports and the like can be faked as well, but this requires a substantial amount of criminal energy.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Transparency</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">: If we analyze existing campaigns, lack of transparency is one of the biggest issues. By „lack of transparency“ I am referring to the fact that backers often know almost nothing about campaign owners. This is related to the previous aspect. While backers need to guarantee with credit card payments that they are trustworthy (which is checked by the credit card companies), they only get a tiny amount of <span class="Apple-converted-space"> </span>information about campaign owners in return. Wait a minute. I am paying my contribution to people that are mostly anonymous (i.e. hiding behind a campaign web site)? Unfortunately, the answer is yes. It does not suffice when only the crowdfunding company owns detailed information about the campaign owners.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Due diligence</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"> measures would require a crowdfunding company to technically check whether a campaign respectively project is feasible. For this purpose, they may hire experts in the respective domain to validate the claims campaign owners make. In addition, they should check the background of campaign owners, be it companies or individuals. <span class="Apple-converted-space"> </span>If a successful company such as Anker acts as the campaign owner, there is a much higher chance that contributors will receive the offered perks and rewards. If on the other hand the campaign originator is unknown, the risk is significantly higher. Accountability should come to one’s mind when thinking about campaigns and their originators.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Check and balances</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">: step-wise transfer of contributions instead of full payment at once. This may be a bit difficult to achieve, because certainly some upfront investments are required by campaign owners. Nonetheless, I’d expect more of a bank (crowdfunding platform)/borrower (campaign owner) attitude in this context. In each step (such as prototyping, testing, final product design, manufacturing, delivery) the crowdfunding company should demand proofs by the campaign owners what they did and achieve so far with the crowdfunding investments. For example, prototyping only requires a smaller amount of money. After coming up with <span class="Apple-converted-space"> </span>a successful prototype, they may move forward to completing the product. After the product is ready, they move further to manufactoring. <span class="Apple-converted-space"> </span>In each step they obtain predefined percentages of funding. In addition, campaign owners are supposed to provide a concrete time line for all of their activities. If a step is delayed, no further money can be obtained until the step is completed. A kind of traffic light on the project web site could represent the current risk level of a campaign.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Shipment</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">: for each project campaign owners need to prove that they actually shipped the perks and rewards to their backers by presenting respective documents from the delivery service. In my experience, some campaign owners marked the perks as being shipped without ever actually sending any items.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">Insurance</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">: Crowdfunding companies should pay a part of each contribution to an insurance company that covers all risks and pays back a high percentage of the contribution to backers. This is similar to how Paypal works. It would require campaign originators to disclose personal information which can then be rated in terms of credibility, credit history, financial background, and trustability. This puts more burden to the campaign owners and the crowdfunding company, and makes contributions more expensive, but provides a safety net for backers which are those who pay campaign owners and crowdfunding platforms, anyway. I assume, many backers would be willing to pay a slightly higher contribution if they win more security in return. Of course, crowdfunding platforms could act as insurances themselves if they are willing to do so.</span></li><li class="li1" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s3" style="font-family: UICTFontTextStyleItalicBody; font-size: 19.08px; font-style: italic;">No selling on other channels</span><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">: In some campaigns the perk developers started selling their products via their web sites before some backers even received their perks. The contract between campaigners and crowdsourcing plaforms should definitely exclude this possibility. Whenever backers spend funding to product development via a crowdfunding campaign, they must be the first who receive their perks and rewards. In addition, some of the products sold were significantly cheaper than the claimed MSRP. This looks like betrayal, smells like betrayal and is a betrayal. <span class="Apple-converted-space"> </span>In such cases I‘d expect campaign owners to have to pay penalties to backers.</span></li></ul><p class="p1" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s1" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Some may argue that all of these measures restrict the freedom of campaign owners. They are right in this respect. However, there currently is an imbalance between contributors, campaign originators, and crowdfunding platforms which puts most risks on the backers. Thus, it seems more than fair to share these risks among all stakeholders. I honestly believe, that crowdfunding evolves to a dead end, if companies like Indiegogo continue to put all burdens to backers, don‘t care much about scams, refuse to create safety nets, or keep the high intransparency. If they realize all or at least some of the aforementioned measures, this clearly will turn out to be more of a Win/Win/Win scenario. </span></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-17867186513912258582023-08-20T12:55:00.009+02:002023-08-20T13:00:43.567+02:00AI and Software Architecture - Two Sides of the same Coin<h3 style="text-align: left;"><span style="-webkit-text-size-adjust: auto; font-family: UICTFontTextStyleBody; font-size: 19.08px;">Introduction</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">The whole media worldwide is currently jumping on the AI bandwagon. In particular, Large Language Models (LLM) such as ChatGPT sound appealing and intimidating at the same time. When we dive deeper into the technology behind AI, it doesn‘t feel that strange at all. In contrast to some assumptions of the yellow press, we are far away from a strong AI that resembles human intelligence. This means, blockbusters such as Terminator or Bladerunner are not becoming true in the near future. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Current AI applications, while very impressive, represent instantiations of weak AI. <span class="Apple-converted-space"> </span>Take object detection as an example, where a neural network learns to figure out what is depicted on an image. Is it a cat, a dog, a rabbit, a human or something different? Eventually, neural networks process training data to compute and learn a nonlinear mathematical function that works incredibly well for making good guesses (aka hypotheses) with high precision about new data. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">On the other side, this capability proves to be very handy when dealing with big or unstructured data such as images, videos, audio streams, time series data, or Kafka streams. For example, autonomous driving systems strongly depend on such kind of functionality, because they continuously need to analyze, understand and handle highly dynamic traffic contexts, e.g., potential obstacles.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">In this article, I am not going to explain the different kinds of AI algorithms such as types of artificial neural networks and ML (Machine Learning) which may be part of a subsequent article. My goal is to draw the landcape of AI with respect to software architecture & design.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">There are obviously two ways of applying AI technologies to software architecture:</span></p><ul class="ul1" style="-webkit-text-size-adjust: auto; list-style-type: "— ";"><li class="li3" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s3" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">One way is to let AI algorithms support software architects and designers in their tasks such as requirements engineering, architecture design, implementation or testing - which I’ll call the AI solution domain perspective.</span></li><li class="li3" style="font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s3" style="font-family: UICTFontTextStyleBody; font-size: 17px;"></span><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">The other way is the use of AI to solve specific problems in the problem domain, why I’ll name it the AI application domain perspective.</span></li></ul><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">AI for the Solution Domain</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">LLMs are probably the most promising approach when we consider the solution domain. Tools such as GitHub Copilot, Meta Llama 2 and Amazon CodeWhisperer help developers generate functionality in their preferred programming language. It seems like magic but comes with a few downsides. For example, you never can be sure whether an LLM learned its code suggestions from copyrighted sources. Nor do you have any guarantee that the code does the right thing in the right way. Any software engineer who leverages an application like Copilot needs to look over the generated code again and again to ensure the code is exactly what she or he expects. It requires software engineering experts to continuously analyze and check LLM answers. At least currently, it appears rather unlikely that laymen may take over the jobs of professional engineers with the help of LLMs. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Companies already have began to create their own LLMs to cover problem domains such as industrial automation. Imagine, you need to develop programs for a PLC (Programmable Logic Control). In such environments, the main languages are not C++, Python or Java. Instead you’ll have to deal with domain-specific languages such as ST (Structured Text = Siemens SCL) or LD (Ladder diagram). Since there is much less source code freely available for PLCs, feeding an LLM with appropriate code examples turns out to be challenging. Nonetheless, it is a feasible objective. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">AI for the Application Domain</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">In many cases Artificial Neural Networks (ANNs) are the basic ingredient for solving problem domain challenges. Take logistics as an example where cameras and ANNs help identity which product is in front of a camera. Other AI algorithms such as SVNs (Support Vector Machines) enable testing equipment to figure out whether a turbine is behaving according to its specification or not, which is commonly coined Anomaly Detection. At Siemens we have used Bayes-Trees to forecast the possible outcome of system testing. Reinforcement Learning happens to be useful for successfully moving and acting in an environment, for example robots learning how to <span class="Apple-converted-space"> </span>complete a task successfully. Another approach is unsupervised learning such as k-Means Clustering which classifies objects and maps them to different categories. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Even more examples exist:</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Think about security measures in a system that comprise keyword and face recognition. Autonomous driving uses object detection and segmentation in addition to other means. Smart sensors include ANNs for smell and gas detection. AI for preventive maintenance helps analyzing whether a machine might fail in the near future based on historical data. With the help of recommender systems online shops can provide recommendations to customers based on their order history and product catalog searches. As always, this is only the tip of the iceberg.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Software Architecture and AI</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">An important topic seldomly addressed in AI literature is how to integrate AI in a software-intensive system. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">MLOps tools support different roles like developers, architects, operators and <span class="Apple-converted-space"> </span>data analysts. Data analysts start with a data collection activity. They may augment the data, apply feature extraction as well as regularization and normalization measures, and select the right AI model which is supposed to <span class="Apple-converted-space"> </span>learn how to achieve a specific goal using the data collection. In the subsequent step they test the AI/ML-model with sufficient test data, i.e. data the model has not seen before. Eventually, they version the model & data and generate an implementation. Needless to say that data analysts typically iterate through these steps several times. When MLOps tools such as Edge Impulse follow a No/Low-Code approach, separation of concerns between different roles can be easily achieved. While data analysts are responsible for the design of the AI model, software engineers can focus on the integration of the AI model in the system design process, as the MLOps envoronment generates implementation of the model.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Software engineers take the implementation and integrate it into the surrounding application context. For example, the model must be fed with new data by the application which reads and processes the results once inference is completed. For this purpose, an event-driven design often turns out to be appropriate, especially when the inference runs on a remote embedded system. If the inference results are critical, resilience might be increased by replicating the same inference engine multiple times in the system. Docker containers and Kubernetes are perfect solutions, in particular when customers desire a scalable and platform-independent architecture with high separation of concerns like in a Microservice architecture. Security measures support privacy, confidentiality, and integrity of input data, inference results and the model itself. In most cases, inference can be treated from a software engineering viewpoint mostly as a black box that expects some input and produces some output. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">When dealing with distributed systems or IoT systems, it may be beneficial to execute inference close to the sources of input data, thus eliminating the need to send around big chunks of data, e.g., sensor data. Even embedded systems like edge or IoT nodes are capable of running inference engines efficiently. In this context, only inference results are often sent to backend servers.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Operators finally deploy the application components onto the physical hardware. Note: a DevOps culture turns out to be even more valuable in an AI context, because more roles are involved.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Input sources may be distributed across the network, but may also comprise local sensor data of an embedded system. In the former case, either Kafka streams or MQTT messages can be appropriate choices to handle the aggregation of necessary input data on behalf of an inference engine. Take processing of weather data as an example where a central system collects data from various weather stations to forecast the weather in a whole region. In this context we might encounter pipelines of AI inference engines, where the results of different inference engines are fed to a central inference engine. Hence, such scenarios comprise hierarchies of possibly distributed inference engines.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Architecting AI models</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Neural networks or other types of AI algorithms expose an architecture themselves, be it a MobileNet model leveraged for transfer learning, a SVN (Support Vector Machines) with a Gaussian kernel, or a Bayes decision tree. The choice of an adequate model has significant impact on the results of AI processing. It requires the selection of an appropriate model and hyperparameters such as learning rate or configuration of layers in an ANN (Artificial Neural Network). For data analysts or those software engineers who wear a data analytics hat a mere black box view is not sufficient. Instead they need a white box view to design respectively configure the appropriate AI model. This task depends on the experience of data analysts, but may also imply a trial-and-error approach for configuring and fine tuning the model. The whole design process for AI models closely resembles software architecture design. It consists of engineering the requirements (goals) of the AI constituents, selecting the right model and training data, testing the implemented AI algorithm, and deploying it. Consequently, we may consider these tasks as the design of a software subsystem or component. If an aforementioned MLOps tool is available und used, it significally can boost design efficiency.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Conclusions</span></h3><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">While the math behind AI models may appear challenging, the concepts and usage are pretty straightforward. Their design and configuration is an important responsibility that experts in Data Analytics and AI should take care of. MLOps helps separate different roles and responsibilities which is why I consider its use as an important development efficiency booster. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;">Architecting an appropriate model is far from being simple, but resembles the process of software design. Training an AI model for ML (Machine Learning) may take weeks or months. As it is a time consuming process, the availability of performant servers is inevitable. Specialized hardware such as Nvidia GPUs or other dedicated NPUs/TPUs helps reduce the training time significantly. In contrast to the amount of required training efforts, optimised inference engines (-> Tensorflow Lite or Lite Micro) often run well and efficient on resource constrained embedded systems which is the concept behind AIoT (AI plus IoT).</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 19.1px; font-stretch: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 24.7px;"><span class="s2" style="font-family: UICTFontTextStyleBody; font-size: 19.08px;"></span><br /></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-63088773187371034502023-04-29T11:14:00.003+02:002023-04-29T11:14:16.897+02:00<h2 style="text-align: left;"> Systematic Re-use</h2><p>Re-use is based upon one of the fundamental principles not only for lazy software engineers: DRY (Don‘t Repeat Yourself). Instead of reinventing the wheel developers and architects may re-use existing artifacts instead of reinventing the wheel again and again. </p><p>Re-usable assets come in different flavors:</p><p></p><ul style="text-align: left;"><li>Code snippets are small building units developers may integrate in their code base. </li><li>Patterns are smart and proven design blueprints that solve recurring problems in specific contexts.</li><li>Libraries comprise encapsulated functionality developers may bind to their own functionality.</li><li>Frameworks also comprise encapsulated functionality. In contrast to libraries developrs integrate their own code into the framework according to the Hollywood principle (don‘t call us, we‘ll call you).</li><li>Components/Services include binary functionality (i.e., they are executables) that developrs may call from their own application.</li><li>Containers represent runtime environments that provide functionality and environments to applications in an isolated way.</li></ul><div>Apparently, these are different levels of re-usable assets with varying granularities, complexities, and prerequisites.</div><div><br /></div><div>Software engineers may not only use re-usable software assets, but other types as well. For instance:</div><div><ul style="text-align: left;"><li>Tests, Test units, Test plans</li><li>Documents</li><li>Production plans</li><li>Configurations</li><li>Business plans</li><li>Software architectures</li><li>Tools</li></ul><div>While some assets such as code snippets may be used daily in the code-base, patterns or software architecture templates need to be instantiated in an easy way. </div></div><div>The more impact re-usable assets have on applications and the more abstract they are, the more systematic the re-use approach must be. The most challenging projects are product lines and ecosystems that require different assets at different re-use levels. For example, they introduce the need for a configurable core asset base that is re-usable across different applications. Furthermore, they support a whole class of applications that share the same architecture framework and other assets. A core asset in a product line or ecosystem affects not one application but a whole system family. Thus, its business impact is very high. </div><div>In such scenarios, core assets often are inter-dependent and must be configured for the specific application under development. As a prerequisite for the development of a core asset base, a Commonality/Variability analysis is necessary that determines what applications sharing the same core assets have in common and how they differ. A core asset needs a common base relevant for all applications that use it as well as configurable variation points to adapt it to the needs of an application. </div><div>A bad or insufficient Commonality/Variability analysis incurs higher costs a may even lead to project failure. </div><div>Core asset development and application development might happen separately by different teams or by the same teams. Each approach has its benefits and liabilities.</div><div>Due to the high business and technical risks of these advanced approaches, all stakeholders need to be involved in the whole development process. Building a product line or ecosystem without management is not feasible. Managers need to re-organize their organisation, spend budget for core asset development and evolution. </div><div>Most product lines and ecosystems fail, because:</div><div><ul style="text-align: left;"><li>lack of management support,</li><li>insufficient consideration of customer needs,</li><li>inappropriate organisation,</li><li>inadequate Commonality/Variability analysis,</li><li>insuffient or low-quality core assets,</li><li>underestimation of testing or inadequate quality assurance,</li><li>bad software architecture,</li><li>neglectence of competence ramp-up activities,</li><li>no re-use incentives,</li><li>missing acceptance by stakeholders.</li></ul><div>Consequently, product lines and ecosystems need a systematic approach for re-use and must involve different types of stakeholders. They need a manager who is able to guide the approach and has the capability to decide, for example, on budget, organisation restructuring, competence ramp-up activities, or business strategy. </div><div><br /></div><div>Re-use comes in different flavors and the higher its impact the more systematic the re-use process needs to be</div><div><br /></div><div>[to be continued]</div><div><br /></div></div><div><br /></div><div><br /></div><div><br /></div><p></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-14229163119794018592023-03-26T15:35:00.001+02:002023-03-26T15:35:07.480+02:00<p> </p><h2 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Models and Modelling - A Philosophical Deep Dive</span></h2><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Motivation</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Not only in software architecture we use models for designing and documenting systems. Models are also indispensible in other engineering disciplines and in natural sciences. We all experienced good and bad models in our daily lifes. What is a model really about? And how does a good model look like? Let us enter a (philosophical) discussion about this topic.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><h3 style="text-align: left;"><span style="font-family: inherit;">What is a model?</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">A model captures the essence of a domain. It focuses on the core entities and the relationships within a domain from a specific viewpoint, i.e., serving a specific purpose. A model contains rules that must hold for its constituents. Models are used by humans or machines to communicate about the respective domain for a particular purpose.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Examples of models include:</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">a UML diagram</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">a street map</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">a floor plan</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">an electronic circuit diagram</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">a problem domain model (DDD)</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">quantum theory</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">mathematical formulas</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Consequences: </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(i) The same domain can be represented using different models, each capturing another viewpoint of that domain. This viewpoints are often briefly called views.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(ii) Models can be informal or formal depending on their usage as a means for communication. Thus, they must be easily understandable and comprehensible by stakeholders.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(iii) Models introduce abstraction layers by using generalization and specialization leaving out „unnecessary“ respectively irrelevant details. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(iv) <span class="Apple-converted-space"> </span>A model does not describe reality but a subset of reality viewed from a specific angle.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(v) Languages are based upon models. A model can be viewed as a language, and vice versa. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">(vi) A model may support a graphical presentation or a textual presentation, it even may include both.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">The complexity of a model is directly proportional </span></p><ul class="ul1" style="-webkit-text-size-adjust: auto; list-style-type: "— ";"><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">to the number and types of its entities and their relationships,</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">to the kinds and numbers of abstractions being used,</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">to the complexity of its underlying rules.</span></li></ul><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">A good model:</span></p><ul class="ul1" style="-webkit-text-size-adjust: auto; list-style-type: "— ";"><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">provides a proper separation of concerns (SoC)</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">consequently applies principles such as the single responsibility principle (SRP), Don’t-Repeat Yourself (DRY), KiSS, or the Liskov Substitution Principle (LSP) in order to gain the highest understandability and comprehensiveness</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">uses expressive names for all its abstractions, entities, dependencies</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">provides an effective and efficient means of communicating among stakeholders</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">focuses on essence and leaves out everything that does not serve the required purpose of the addressed viewpoint</span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">avoids accidental complexity strictly and consequently </span></li><li class="li2" style="font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">allows to model simple things in a simple way, while being capable of expressing complex things in a doable way</span></li></ul><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><br /><span class="s2" style="font-family: UICTFontTextStyleBody;"></span></p><h3 style="text-align: left;">Stakeholders</h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">The creation of a model should be guided by its (types of) stakeholders, in particular by the way they intend to use the model. In this context a meta model helps define how the set of creatable models should look like. Thus, meta models constitute modeling languages. They help create different models or views.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">To define an adequate model that serves an intended purpose all (human) stakeholders should be involved. UML is an example of a modelling language that serves the needs of software engineers but (often) not those of many domain experts. In fact, domain experts might have their own models readily available. While a model might be perfect for machine-machine communication, it aint’t necessarily adequate whenever humans are involved. The more formal a model is, the easier it can be processed by computers. Humans often need more informal and expressive models instead. If both kinds of stakeholders are involved, we need to balance between formal and informal approaches. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Emojis are an example of an informal model. They can be immediately understood by a human, but may be more difficult to process by a machine.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Artificial Neural Networks albeit “simple” can be processed by machines very well, but are hard to be understood by a human - i.e., with respect to what they actually do and how they work. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">UML is somewhere in the middle of these extremes. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Fortunately in many mature domains, models already exist. An electrical circuit defines a proven concept of a model. Mathematics is often considered a uniquitous language with predefined notations. In the context of software engineering, domain models are often implicitly defined and have been established as common sense in an organization. If software engineers with no or little domain expertise start to develop software applications for the respective domain, they need to make the implicit model explicit. Otherwise they cannot design a software architecture that meets the customer requirements. This is what DDD (Domain-Driven Design) is all about. It tries to come up with a domain-specific model using generic building blocks such as DDD patterns and techniques.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">The representation of a model should fit the needs of its stakeholders. For humans graphical notations often work very well, because they explicily reveal their structure in an easy manner and are good to grasp and to handle. Due <span class="Apple-converted-space"> </span>to productivity reasons, textual models may be more beneficial and flexible in some cases. As an example consider software code. For a beginner graphical code blocks might work very well, while advanced programmers prefer coding textually, because they can mentally map seamlessly between the “graphical” design and the textual code representation. Handling code graphically might just reduce their productivity, effectiveness and flexibility due to all clutter and constraints.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Model Transformations</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">To keep many stakeholders satisfied a possible approach might be to introduce different models for different types of stakeholders and also create mappings between these models, for example an easy to understand UML model that is transformed into a machine readable XML schema. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Actually, software engineers are used to handle different models that are mapped onto each other. In software engineering a compiler represents a model transformation from a high level language to a system language or interpreter. A UML diagram might be transformed into high level language code. A low-code/no-code environment creates domain-specific applications from high level user specifications. However, model-to-model transformations can be quite complex, in particular when the gap between models is very large and if no common-off-the-shelve solutions for the transformations are available. Moreover, the more models the more transformations are necessary. Note: a model transformation might also be done manually if the model is not too complex and mapping rules are pretty straightforward.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Model sets</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">In domains such as building contruction or software engineering multiple views are necessary to represent information from different angles. Take design view, deployment view or runtime view as examples in the software engineering domain. In addition, their might be different model abstraction layers, for example, an in-depth design view versus a high level software architecture view. <span class="Apple-converted-space"> </span>In other words, to solve a task we need a model set instead of a single model that captures every detail from every perspective.</span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">No matter how the views differ from each other, there needs to be meta information to tie the different views together. Prominent examples are the mapping from a view to code, and the implicit or explicit relation of views with each other. Note: there might be different solutions respectively model kits for the same problem context, e.g., RUP’s 4+1 view in contrast to TOGAF might not be the (only) solution of choice for designing an enterpise system. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">No matter what model set you choose, make sure that it is used consistently. In most cases tool support is strongly recommendable. Models can become very complex. Therefore you need a tool to draw, check and communicate the concrete models. This is the main reason why most software engineering activities rely on some sort of UML environment such as Enterprise Architect or MagicDraw. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">A ground plan is different from an electricity plan. All models together are necessary for building construction. <span class="Apple-converted-space"> </span>In this example, there might <span class="Apple-converted-space"> </span>also be rules and constraints across all models respectively views. For example, an electrical cable should have a minimum distance to a water pipe. Consequently, we need some kind of verification algorithm to check whether rules/constraints are violated. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Model Creation</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Models shall never be created in a big bang approach. They are living entities that change over time the more experience you obtain. They may start very simple but become more complex over time. Whenever they are overengineered, they need to be simplified/refactored again. Model creaters need to ensure that models can be facilitated and handled by stakeholders easily. If stakeholders have different viewpoints at the same problem, create a model set where each model view serves a particular set of stakeholders.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">To start creating a model for a domain context, we should figure out whether such models already exist, and if this is the case, whether these models can serve the desired purpose(s). It is always beneficial to use existing models, in particular due to the experience and knowledge they carry. So, don‘t reinvent the wheel if not absolutely necessary, especially if you are no expert in the domain.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">If no model exists, stakeholders should jointly create a model (set). It is helpful if at least one of the stakeholders is experienced in creating models while at least some other person is a domain expert.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">If models exist that do not serve the intended purpose, we might change and adapt these models to fit our needs.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Note: a common mistake is to first focus on the syntax of a model. Instead, initially think about its semantics and find a good syntactical representation afterwards. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">No matter how a new model is created, <span class="Apple-converted-space"> </span>learning its representation should happen in a quick and straightforward process, even for unexperienced stakeholders.</span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Interestingly, most graphical models consist of rectangular or other symmetrical shapes, arrows, lines and textboxes, while textual models often use regular or context-free grammars. The reason for this observation is that this way the models are comprehensible and their handling is easy. It should also be possible to draw a model manually in order to discuss it with other stakeholders before documenting it. <span class="Apple-converted-space"> </span>Sitting around a modelling tool significantly decreases productivity, at least in my experience. A whiteboard or a flipboard is by far the best tool for modelling. This can be complimented by an AI software that recognizes manually drawn models and transforms them to clean and processable data representations. </span></p><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"><br /></span></p><h3 style="text-align: left;"><span class="s2" style="font-family: UICTFontTextStyleBody;">Summary</span></h3><p class="p2" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px;"><span class="s2" style="font-family: UICTFontTextStyleBody;">In this blog posting I did not reveal any new or innovative stuff you didn‘t already know. Neither was it my intent to provide anything revolutionary. It is just a summary of modelling and how to approach it. And if you started thinking about modelling from this more philosophical view, I‘d be happy. </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"> </span></p><p class="p3" style="-webkit-text-size-adjust: auto; font-size: 17px; font-stretch: normal; line-height: normal; margin: 0px; min-height: 22px;"><span class="s2" style="font-family: UICTFontTextStyleBody;"></span><br /></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-36201392718076558842020-10-21T13:04:00.001+02:002020-10-21T13:04:22.732+02:00<h1 style="text-align: left;"> Cohesion and Coupling</h1><div>Sometimes what appears to be very simple and clear, is not that clear at all. A prominent example is the difference between <b>cohesion</b> and <b>coupling</b>.</div><div><ul style="text-align: left;"><ul><li><b>Cohesion</b> indicates the relationships between components in a module. These relationships are not just syntactic sugar but have a semantic implication. For example, if a module provides different separate functionalities it will reveal low cohesion. In constrast, modules with high cohesion implement functionalities that are tightly interweaved with each other. In other words, modules with a low cohesion provide different things that are not related with each other. Thus, they violate the SRP principle (SRP = Single Responsibility Principle). The best refactoring in such cases is to split up the module into different modules that obey the SRP.</li><li><b>Coupling</b> defines the number of relationships between different modules. A module A that introduces a lot of relations to module B, strongly depends on module B, and vice versa. Strong respectively tight coupling of modules indicates that functionalities in B could be integrated in A. The obvious refactoring would integrate all functionalities of B into A, thus decreasing coupling of the system. Of course, merging of modules might reduce cohesion in the resulting module A. But what if another module A* also depends heavily on B. Should we then merge A and B as well as A* and B? Or should we merge A and B and reroute (A*, B) relations to A, thus possibly increasing coupling between A and A*? The best option is to strive for high cohesion and low coupling throughout our system and its constituting modules. Merging B into A and A* at the same time might violate the DRY principle (Don‘t repeat Yourself). But as we know, the DRY principle is not always appropriate. In Microservices architectures the diffenent microservices should hide their implementation details and do not share the same information but be independent of each other. Nonetheless, the principles of high cohesion and low coupling still apply.</li></ul></ul><div>Fortunately, architecture introspection tools can visualize cohesion and coupling and even provide hints and mechanisms how to restructure a given system. </div></div>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-51061714537679500672019-01-18T13:08:00.000+01:002019-01-25T22:16:06.058+01:00IoTDeepDive<h2>
IoTDeepDive - Infos zum OOP 2019 Ganztagstutorium Fr4</h2>
Diese Webseite ist auch erreichbar über folgende URL: <a href="https://tinyurl.com/oop2019-iotdeepdive">https://tinyurl.com/oop2019-iotdeepdive</a><br />
<h3>
Hardwareanforderungen</h3>
<ol>
<li>Windows, Mac, Linux</li>
<li>Der Rechner benötigt WLAN-Fähigkeit über WPA/WPA2. WLAN-Verbindung stellt der Veranstalter bereit.</li>
<li>Ein externer USB-Port sollte vorhanden sein.</li>
<li>Zum Zugriff auf den seriellen Port mit dem auf dem Board verwendeten Silicon Labs USB-zu-UART, ist ein <a href="https://www.silabs.com/products/development-tools/software/usb-to-uart-bridge-vcp-drivers">Treiber</a> nötig. <span style="text-align: center;">Bitte Version für eigenes Betriebssystem herunterladen und installieren!</span></li>
</ol>
<ul>
</ul>
<h3>
Softwarevoraussetzungen</h3>
<ol>
<li>Im Tutorium kommt die Arduino IDE zum Einsatz. Download über <a href="https://www.arduino.cc/en/Main/Software">https://www.arduino.cc/en/Main/Software</a> und Installation gemäß der dortigen Beschreibung (abhängig vom Betriebssystem).</li>
<li>In der Arduino IDE muss ein zusätzlicher Boardsmanager für ESP32-Boards installiert werden. Für eine Beschreibung des Vorgehens siehe: <a href="http://esp32-server.de/">http://esp32-server.de/</a></li>
<li>Adafruit-Bibliotheken in der Arduino IDE installieren, falls nicht über Library Manager verfügbar: </li>
<li>Allgemeine Sensorbibliothek <a href="https://github.com/adafruit/Adafruit_Sensor">hier</a>. Als Voraussetzung für die nachfolgenden Bibliotheken wichtig. Jede Bibliothek <span style="background-color: #e0e0e0; color: #333333; font-family: "verdana" , sans-serif; font-size: x-small;">per .ZIP Download holen (GitHub Clone or Download) und dann mittels Sketch>Include Library>Add.ZIP Library der IDE hinzufügen.</span></li>
<li>DHT 11: <a href="https://github.com/adafruit/DHT-sensor-library">hier</a>. </li>
<li>BME680: Die Bibliotheken von Adafruit funktionieren auch für das Watterott-Breakout. Zugehörige BME680-Bibliothek <a href="https://github.com/adafruit/Adafruit_BME680">hier</a> herunterladen. </li>
</ol>
<h3>
Source Code für Beispiele</h3>
<div>
Die Quelldateien für die Übungen finden Sie <a href="https://www.dropbox.com/s/vpm7tteetozybgc/IotDeepDiveBeispiele.zip?dl=0">hier</a></div>
<h3>
Handouts</h3>
<div>
Bitte <a href="https://www.dropbox.com/s/huv0yn625kiw9fb/oop2019-Fr4-IoTDeepDive.pdf?dl=0">hier Handouts als PDF herunterladen</a>.<br />
<a href="https://www.dropbox.com/s/i1qsdb5omzbzumt/oop2019-Fr4-IoTDeepDiveColor.pdf?dl=0">Handouts</a> in Farbe.</div>
<div>
<br /></div>
<h3>
IoT Hardware Bezugsquellen</h3>
<div>
<ul>
<li>ESP32 Entwicklungsboard im <a href="https://www.makershop.de/plattformen/nodemcu/espressif-esp32-dev-kit-board/">Makershop</a>.</li>
<li>BME680 Sensor (Wetter/Umwelt) Breakout-Board bei <a href="https://www.watterott.com/index.php?page=product&info=5221">Watterott</a>.</li>
<li>DHT11 (Temperatur/Feuchtigkeit) bei <a href="https://www.exp-tech.de/sensoren/temperatur/8998/dht11-temperature-humidity-sensor">exp-tech</a>.</li>
<li>DHT22 (genauerer Bruder des DHT11) bei <a href="https://www.exp-tech.de/sensoren/temperatur/8999/dht22-temperature-humidity-sensor">exp-tech</a>.</li>
<li>Breadboard bei <a href="https://www.exp-tech.de/zubehoer/breadboards/5031/breadboard-830-630/200">exp-tech</a>. Besser, aber auch teurer, sind freilich Labor-Breadboards wie das von <a href="https://www.pollin.de/p/labor-steckboard-510177">pollin</a>.</li>
<li>Jumper (Dupontkabel) zum Beispiel auf <a href="https://www.amazon.de/Female-Female-Male-Female-Male-Male-Steckbr%C3%BCcken-Drahtbr%C3%BCcken-bunt/dp/B01EV70C78/ref=sr_1_3?ie=UTF8&qid=1548450241&sr=8-3&keywords=dupont+kabel">amazon</a>.</li>
<li>LEDs zum Beispiel als Sortiment auf <a href="https://www.amazon.de/KINGSO-Leuchtdioden-Dioden-Elektronik-komponenten/dp/B01KVH6NA2/ref=sr_1_2?ie=UTF8&qid=1548450375&sr=8-2&keywords=led+sortiment">amazon</a>.</li>
<li>Widerstandssortiment ebenfalls über <a href="https://www.amazon.de/Elegoo-Widerst%C3%A4nde-Sortiment-St%C3%BCck-Metallfilm/dp/B072BHDBDG/ref=sr_1_3?ie=UTF8&qid=1548450448&sr=8-3&keywords=widerstand+sortiment">amazon</a>.</li>
<li>Anschlusskabel Micro USB auch auf <a href="https://www.amazon.de/CSL-Metallstecker-Nylonmantel-strapazierf%C3%A4hig-vergoldeten-Wei%C3%9F-Schwarz/dp/B010BKWOCK/ref=sr_1_27?ie=UTF8&qid=1548450661&sr=8-27&keywords=micro+usb+kabel+1m">amazon</a>.</li>
</ul>
<div>
Wer sparen will, versucht in China (ebay, Banggood, Alibaba) entsprechende Komponenten zu beziehen. Dauert of lange und ist eventuell mit einem Gang zum Zoll verbunden.</div>
</div>
<ol>
</ol>
<ol>
</ol>
Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com2tag:blogger.com,1999:blog-5119025.post-75264939417234302602015-08-16T00:39:00.001+02:002015-08-17T17:26:01.771+02:00What the hell is Software ArchitectureSince the Nineties many definitions of software architecture have been proposed, most of them being very vague. Eventually, the common denominator of these definitions suggests that software architecture denotes a set of cooperating components. In my opinion, this view is rather simplistic. Most entities in the universe follow the same definition which makes the definition useless from an engineering perspective.<br /><br />Instead of providing yet another definition of software architecture we should think about its properties.<br /><br />(i) Software Architecture is both a process and a thing: the process of architecting comprises a sequence of strategic decisions, while the thing is the result of this process. The goal of software architecture is to create the backbone for implementing the specification. Note, that we do not assume a specific software development paradigm here such as Lean or Agile Development. <br /><br />Remark 1: Strategic decisions refer to requirements and additional forces that affect the whole architecture and result in design artifacts that are tightly coupled with the rest of the architecture which makes them difficult and expensive to change. Examples include mandatory functional properties of the system, operational qualities such as performance or security, infrastructures for modifiability such as a Plug-in Architecture, constraints caused by the system context such as hardware prerequisites.<br /><br />Remark 2: All architecture design decisions must be driven by the specification and the business goals. Put briefly: No decision without a (good) reason.<br /><br />(ii) Architecture design spans the whole lifecycle of a system not just its creation. It starts with initial planning and requirements engineering and ends when the system(s) built upon this architecture reach their end. Between these various points in time it covers creation, maintenance and evolution. <br /><br />(iii) In a naive sense every software-intensive system reveals a software architecture, even if it has been created in a complete unsystematic or unintentional way, i.e., using ad-hoc decisions. What we need is systematic architecture design driven by well defined and prioritized architecturally-relevant requirements as well as risks. Sometimes, some or even all parts of an existing system with an unknown or partially known software architecture need to be (re-)used. In this case software archeology methods are required to help extract the hidden software architecture and make it explicit.<br /><br />(iv) There are two kinds of architecture quality, external quality and internal quality. While the former one defines the externally visible behavior as demanded by quality attributes, the latter defines the habitability of architectural artifacts (such as simplicity or expressiveness) by developers, testers, etc.<br /><br />Remark: a consequence of habitability is the limitation of software architecture design to a small number of hierarchical entities such as system, subsystem, components. This entities should follow the Single-Responsibility Principle. Accordingly, the responsibility of fine design and implementation is to refine and extend these abstractions to provide executable artifacts.<br /><br />(v) For architecture as a process a consistent set of guidelines and tools shall guide the architecture design in order to ensure high quality and business alignment. Without such guidelines the architecture will be overloaded with multiple styles, idioms, patterns, concepts, technologies, paradigms, conventions, all of which reduces internal quality. <br /><br />Remark: One major challenge is taming inherent complexity while avoiding accidental complexity. The latter one can be caused by using the wrong solutions, applying the right solutions incorrectly. <br /><br />(vi) Software architecture as a thing is a means for communicating design decisions to other stakeholders. Thus, all decisions must be made explicit in a comprehensive way. The various constituents of the architecture shall be provided to readers in adequate ways depending on their roles and goals and their responsibilities and expectations. For this purpose, software architecture documentation must offer a consistent and complete set of architectural views.<br /><br />Remark 1: Since software architecture is a thing and a process, its documentation includes the sequence of design decisions and their rationale which relates architecture views and process and which enables requirements and decision traceability.<br /><br />Remark 2: Software architecture creates a base for design & implementation. It defines an initial (walking) skeleton for deriving the code base. This is why habitability is of foremost importance.<br /><br />(vii) A software architecture is not an island but embedded into a context. Thus, it is important to strictly separate the architecture from its environment, while at the same time considering and defining the interfaces and interactions between both. Otherwise, it won't be possible to come up with an appropriate software architecture. A context view and use case views are examples that help address these aspects.<br /><br />(viii) Software architecture must cover both problem domain and solution domain. For this reason, a Multi-Tier design is not an architecture. To avoid monolithic designs both domains should be hierarchically structured into subdomains and their relationsships. The organization of software architecture activities shall be driven by the aforementioned subdomains, not by the line organization. With other words, mind Conways law!<br /><br />(ix) Software architecture is not necessarily constrained to a single system. It might also define the base for a set of systems within a specific problem domain. In such reuse contexts a Commonality/Variability analysis is required to define a common base architecture which will be modified for a particular implementation context before fine design and implementation start.<br />Examples: product lines, ecosystems, platforms/infrastructures, libraries.<br /><br />Remark: This is what reference architecture and product-line architectures are about.<br /><br />(x) Architecture design must follow a test-driven approach and communicated in such a way that the architecture can be tested. Testing provides relevant information to the architects such as revealing quality issues or design flaws. It includes quantitative and qualitative architecture reviews.<br /><br />I can't resist. Let me conclude this posting with yet another definition of software architecture:<br /><br />Software Architecture <br />is a process, i.e., a sequence of intentional strategic design decisions which map specification and business goals to architecture design. <br />is a thing, i.e., a set of views that address different stakeholders and are the results of the architecture process.<br /><br />The main challenge for software architects is: there are different ways to design a good software architecture, but there are infinite ways to create a bad one.<br /><br />Addendum July, 17th<br /><br />Some may wonder why software architecture should be considered a process as well. <br /><br />It is insufficient to obtain some architecture views, i.e. the What. In addition we might need information how the architecture has been created and why it is as it is, i.e. the How and the Why. This is not essential for some stakeholders such as users, but it provides relevant information to developers, customer service, and testers. <br /><br />One notable example is the detection of design flaws. When we encounter a flaw in our system, we'd like to know when the design flaw has entered the "crime scene" and what other decisions depend on this flawed decision. This gives us traceability and the possibility to rollback in a systematic way. Likewise, process knowledge is important for architecture reviews.<br /><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com4tag:blogger.com,1999:blog-5119025.post-27489680686482382222015-05-13T19:45:00.001+02:002015-05-13T19:50:27.234+02:00A Matter of WasteIf a queue has a capacity of 100 units, we may enqueue 10 entities with a volume of 10 units per entity. 10 times 10 equals 100, right?<br /><br />If we find a parking space with exactly the same length as our car, our search will come to an end - assuming the parking car must be parked parallel to the sidewalk. Right?<br /><br />Hmmmh, the last one does not feel right. But what is the problem? It is that we need some additional space to maneuver. This extra space could be considered as waste, but it is in fact a precondition for parking operations. We may call this quality attribute "parkability".<br /><br />A similar problem is the issue of a rectangular container with a volume of 100 ought to be filled with 100 spheres of volume 1 each. Obviously, we can't expect 100 spheres to fit into a container with the size of all spheres summed up. Again, the challenge is caused by inherent extra space needed by spheres.<br /><br />Do we encounter such challenges in software design as well? Of course! <br />To be precise, we experience "waste" in two different areas: in our own activities as engineers and in our systems.<br /><br />Let me start with the latter one that is primarily caused by technology and design constraints as well as by inherent properties of the problem domain. If you got this cool new 64-core machine with Terabytes of RAM, you may want to increase the efficiency of your apps by 64 times. Unfortunately, waste comes into our way again. Competition for resources and inherent dependencies in the problem context will cause the performance increase to be significantly lower than the desired factor of 64. And we did not even consider accidental complexity in this scenario.<br /><br />Interestingly, it is such dynamic overhead that makes it difficult to deal with operational qualities or to build realistic simulators.<br /><br />But how does this waste issue relate to software engineering activities? Basically, it is the same calculus again.<br /><br />If we are committed to two different activities with overlapping time spans that need our attention, there is a chance of conflicts. Two ressources are competing for your time. As you cannot focus on two activities at the same time, you'll have to focus on one, which leads to a kind of time debt in the other one.<br /><br />Another approach is switching back and forth between two activities. However, this requires the engineer to stay synchronized with the current state of each activity. Experts in multithreading call this "context switch". Time needed for such housekeeping duties inevitably causes waste.<br /><br />Another typical root of waste is when a consultant or engineer of a company is supporting a custumer, because it is necessary to deal with two organizations such as organizational overhead for vacation, payments, ordering items, time accounting, meetings.<br /><br />Often engineers do not consider such overhead, because they underestimate the problem instead of making waste explicit and finding ways to reduce it. But keep in mind: It is impossible to remove all waste, in particular inherent one.<br /><br />Experience tells us that at least (!) one fifth of time allocated to an activity will be consumed for "non productive" issues. Thus, we always need to explicitly add a waste factor representing idle time.<br /><br />Another reason for waste is communication. Think about reading mail, joining meetings, handling phone calls, chating with colleagues, drinking coffee. In a magazine I recently read that such interrupted work is a major issue in companies causing incredible amounts of overhead. This is due to the fact that when humans are experiencing a non maskable interrupt they'll need to reboot their brain before refocusing and continuing with the interrupted work.<br /><br />So, if you need to reduce productivity of colleagues significantly play the interrupt game. Tom DeMarco once told me about two printer companies in the US where he could relate meeting time and productivity. The more (mandatory) meetings the less productive a company will be. Maybe, this pattern could be called "death by meetings". If you appreciate further patterns, start reading Dilbert by Scott Adams.<br /><br /><br /><br />- Posted using BlogPress from my iPad<br /><br /><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-27131732243575234212015-03-05T22:35:00.001+01:002015-03-05T22:49:43.319+01:00Technical Debt - The Downside of MetaphorsThe term "debt" is a metaphor from economy. Its use for software development seems to be very reasonable. Whenever developers fail to address quality issues in their software, they will have to pay this debt back. That is quite simple, isn't it? <br /><br />However, we can easily find weaknesses of the term "technical debt":<br /><br />When a system reaches its end of lifecycle, all debt will be gone. Try this for financial debt.<br /><br />Financial debt is a separate entity, i.e. two debts do not intertwine, while software debt sometimes can't be easily located, isolated or separated, because it is woven into the system.<br /><br />Technical debt may stay unpaid, if software engineering comes to the conclusion that the cost for refactoring would be higher than keeping the debt untouched.<br /><br />Financial debt is created intentionally. This is certainly true for some technical debt issues as well such as temporary workarounds. However, many kinds of technical debt are accidental without architects even recognizing it.<br /><br />Technical debt is destructive. If possible, you would like all of it to be eliminated. In contrast, financial debt often is more constructive - such as increased cash flow. <br /><br />Financial debt is independent of other system parts, while technical dept in one place can be affected by other parts, and vice versa. An example would be modification of the system that makes the code parts containing the technical debt obsolete.<br /><br />Interest rates are mostly determined by banks and usually stay fixed over the specified time. In the context of design debt the interest rates increase the longer the debt is not treated. The pay back time is usually not predetermined. And the same kind of debt may be more expensive to pay back in one system part than in other parts. <br /><br />While almost all forms of financial debt are paid back continuously until the debt plus the interest rates are fully covered, each single technical debt in most cases must be paid back at once. <br /><br />There is one kind of debt in the financial domain, while technical debt has many different shapes. <br /><br />If you think about it, you'll find other metaphors that are more convincing. For example, a tumor or infection has many similarities with what we call technical debt: It can occur accidentally, or intentionally if you don't take care of your health. It must be paid back at once - you are either ill or not, which isn't quite true in practice, but very close. The longer it is not treated, the worse the health condition will become - think of viruses that spread and damage other organs. Infections and technical debt are mostly destructive. Both can be monitored in imaging devices or labor diagnostics. There are tumors that are difficult to get rid of and that cause severe damage, while others are harmless if treated early. The worst ones can even become lethal. So my first guess would be to call it <b>software infection</b> instead of software debt.<br /><br />Another possibility is to use environmental pollution as a metaphor. Like technical debt it may occur incidentally or (most of the time) accidentally. You must pay it back, but you can do this part by part. If left untreated, the situation worsens. It is purely destructive. Often, it can't be easily separated from the environment. Thus, another possibility would be to call it <b> software pollution</b>. Software Architecture Sustainability is a metaphor that is related to environmental issues. And so is the term "Software Ecosystem". <br /><br />Other metaphors could be considered as well such as terms for similar issues in hardware or building architecture (consider <b>design erosion</b>).<br /><br />My personal favourite is Software Infection (respectively technical infection, code infection, design infection). Of course, this metaphor has its own liabilities, but it is much closer to its cousin in software development than the debt metaphor is.<br /><br />And it sounds appropriate to speak about a sick component or architecture. Or to visualize an infection using a modality like in medicine.<br /><br />I know that it is too late to get rid of the expression "technical debt", but at least we should handle it with care. Some metaphors that sound good in the beginning may turn out to be not well aligned with what we like to visualize. <br /><br /><br /><br /><br /><br /><br /><br /><p class='blogpress_location'>Location:<a href='http://maps.google.com/maps?q=Auenstra%C3%9Fe,Munich,Germany%4048.126734%2C11.572198&z=10'>Auenstraße,Munich,Germany</a></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com4tag:blogger.com,1999:blog-5119025.post-27596157740470603222015-02-28T13:36:00.001+01:002015-03-01T01:23:24.477+01:00Are Patterns like Mummies?In 1994, the Gang of Four, Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides published their groundbreaking book on design patterns. At almost the same time we were creating the first POSA book (Pattern-Oriented Software Architecture) which was eventually published in 1996. The software engineering community got flooded by the Pattern Wave. And soon there arrived an inflation of pattern books, some of them excellent, but many of them of mediocre quality. <br /><br />I remember, that almost every software development magazine addressed patterns regularly and enthusiastically in the Nineties. The GoF became the Beatles of architects and developers. And most experts anticipated a myriad of new patterns rising at the horizon.<br /><br />Actually, several pattern books have been published until now, but without the impact the GoF had. If you ask software engineers about the patterns they know, almost all will mention GoF patterns, many may illustrate some POSA patterns as well, and only a few will come up with other patterns such as those in Martin Fowlers book on Enterprise patterns.<br /><br />This could suggest, that no other important patterns can be found in software habitats anymore, because GoF already got them all. But even if this were the case for general purpose design patterns, shouldn't there be some excellent patterns lurking in more specific domains? Pattern experts have tried to come up with complete pattern languages for their domains. In theory, such pattern languages are beneficial. In practice, it is impossible to cover a medium to large-sized domain with a pattern language because of the inherent complexity involved. Thus, it is not surprising that existing languages cover only tiny domains or fail to completely cover larger domains.<br /><br />Does the whole universe only know the GoF patterns and that's it? In this case the seminal GoF book would be the holy grail of software development. If not, where do all the unknown patterns hide? <br /><br />Let us assume we are asked to improve the GoF book, what exactly would we change. Patterns like the Null Object Pattern or the Extension Interface pattern (see POSA Vol. 2) could be added, the Singleton pattern removed and other patterns such as Abstract Factory improved. Patterns are not carved in stone but subject to future changes, albeit not with a high evolution speed.<br /><br />If you look at patterns from 30000 feet, you'll recognize that there are some benefits of patterns not related to coding.<br /><br />Pattern forms are a good mental tool to document design. For example, they comprise a name, a context, a problem with forces, and a solution. This helps document all kinds of architecture decisions in a structured way<br /><br />Patterns are also applicable to document best practices for transformations, data representations, and many other topics. For example, each refactoring can be considered a transformation pattern. Best Practices are ideal candidates for patterns<br /><br />Patterns introduce an idiomatic viewpoint, as they define a language. Effective usage of software platforms in terms of best practices for frameworks, libraries, APIs and protocols are idiomatic as well. Experience shows that idiomatic approaches help better understand structures and concepts. In software engineering all structures are idiomatic. Languages change and so do Patterns.<br /><br />The value of patterns is not their content, but also their usage as good mental tools, with the capability of addressing all activities. Patterns help share best practices with others. They unfold a language of idioms that provide understandability, maintainability.<br /><br />Patterns are dead. Long live patterns.<br /><br /><br /><br /><br />- Posted using BlogPress from my iPad<br /><br /><p class='blogpress_location'>Location:<a href='http://maps.google.com/maps?q=Auenstra%C3%9Fe,Munich,Germany%4048.126718%2C11.572199&z=10'>Auenstraße,Munich,Germany</a></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-38993434049653621172015-02-24T17:41:00.001+01:002015-02-24T17:41:46.800+01:00Architecture-as-a-ServiceRecently, I discussed architecture design with some colleagues who are involved in Product Line Engineering projects. When the term "Reference Architecture" came up, it became obvious after a while that they used the term specifically for Product Line Architecture, while I had Reference Architectures in mind, which are typically created to steer and guide standardization. They might also present architecture styles that are commonly accepted within a domain. Think of compilers which are in most cases structured as a Pipes & Filters architecture which is composed of a lexer, a parser, a semantic evaluator, an optimizer, and a code generator. <br /><br />A Reference Architecture (RA), on the other hand, is a coarse grained architecture template different organizations use for guiding their design activities. You may remember the OSI 7 Layers model or the OMG Reference Architecture for CORBA, both invented in the Stone Age of software engineering. A RA is a blueprint which doesn't include any further assets but documents. In fact, it is not very specific but rather abstract.<br /><br />A Product Line Architecture (PLA) is an architecture for the product line of an organization. It captures the commonalities of a set of similar products and defines variation points using feature models or meta models. It contains different core assets, some of which are ready for use. A PLA is much more specific than a RA and defines a framework with (partially) implemented artifacts and variation mechanisms.<br /><br />An RA can be used as the core of a PLA. For this purpose, the RA is concretized in Domain Engineering by addressing requirements and constraints of an organization. In the process, engineers may provide additional variation points or bind existing variation points. In the latter case the PLA transforms a variation point to a commonality. If many organizations do this for the same variation point in similar ways, a new extended RA is born.<br /><br />A PLA can become the base of a RA if it is subject to abstraction and considered as an established practice in the domain, i.e. if most PLAs will end up in the same RA.<br /><br /><br />- Posted using BlogPress from my iPad<br /><p class='blogpress_location'>Location:<a href='http://maps.google.com/maps?q=Wotanstra%C3%9Fe,Munich,Germany%4048.143514%2C11.502871&z=10'>Wotanstraße,Munich,Germany</a></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-27000823795942108842015-02-01T15:17:00.001+01:002015-02-24T16:46:07.743+01:00I am backAfter a long time absence due to health issues I am finally back. I will add new content in the following months with the publication frequency increasing over time.<br /><br />At the OOP 2015 conference which ended last friday I was in charge of the architecture track. In my own talks I covered internal & external quality as welll as ways to ruin a project by (wrong) architecting. The latter tutorial introduces failure patterns for harming software development from the perspective of architects. Unfortunately, there are uncountable ways for failure, but only few for success. <br /><br />A software system may be considered as a living organism. If the organism is not robust enough, even small illnesses may cause large damage. Architecture is basically the infrastructure necessary for metabolism, neural transmission or structural integrity (such as the bones). If parts are damaged, e.g., organs, we may either put in new or artificial organs, or maybe only repair a small part.<br />The software organism is created in an evolutionary process with biological patterns and is subject to design erosion during its whole lifecycle. Obviously, it is important to check the organism regularly to identify potential health problems as soon as possible. With Architecture Tomographs these checks can be fast and intense. In some cases design erosion or cancer dissemination is high enough to make the system impossible to repair. In these situations we may better create a new organism. However, for less harmful kinds of design erosion we may refactor using patterns. In this context, external quality is every property we can observe from outside such as speed, reliability, ... Internal qualities comprise heart rate, blood pressure, insuline level, bone strength, ....<br /><br />Of course, the model has its limitations, but for me it turned out to be really useful.<br /><br /><br />- Posted using BlogPress from my iPad<br /><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-83964522655202640712014-07-14T18:12:00.003+02:002015-02-24T16:45:57.573+01:00Micro Management for Micro Brains - why Micro Services suck It sounds like a great idea. Instead of building a monolithic enterprise application, just split functionality into small components and run them independently in their own processes. These micro services need to cooperate with each other to provide more advanced functionality. For that purpose, they are going to communicate with other micro services.<br /><br /><br /><br />Such an approach promises better flexibility in changing and deploying systems. If only a small part of the functionality is changed, this will affect only a small number of micro services. Or, as we like to say, "small is beautiful".<br /><br /><br /><br />What a wonderful new architecture style. Eventually, we found the silver bullet. Agreed, micro services may not be a silver bullet, but, at least, they are silver shotgun shells.<br /><br /><br /><br />This idea is not new, though. For example, "actors" are providing the same kind of solution. They encapsulate fine grained services behind message-based interfaces that enforce argument and result types to be value types, so that no side effects may occur. In order to achieve a common goal, actors interact with each other. <br /><br /><br /><br />"Micro services" contains the term "services" for a reason. Micro services basically introduce a stripped down resurrection of SOA.<br /><br /><br /><br />However, the delicious looking and tempting apple may be poisoned. There are several "challenges" when using this architecture style:<br /><br /><ul><br /><li><b>Complexity</b>: If we are building a non-trivial application, it will reveal inherent complexity. For the moment, let us assume, there is no accidental complexity involved. When this application is based on micro services, where does the complexity go? Unfortunately, it does not go away, but manifests itself in complex connection and cooperation patterns between tiny services. </li><br /><li><b>Modifiability</b>: If our enterprise application is partitioned into small micro services, we only need to touch and redeploy small services for changing the system, instead of facing the mess of application monoliths. In theory, this is a perfect solution. In practice, it is not, because for non-trivial changes of micro services, we may need to rewrite a lot of other micro services that are connected with the modified micro services. Remember, the complexity does not disappear but shows up in the topology of the micro services network. This does not only affect design but also evolution, assessment and refactoring activities.</li><br /><li><b>Internal Architecture Quality</b>: Architecture is about strategic design. Its main purpose is to map the problem domain to the solution domain. If micro services are used as the primary concept for structuring functionality, the problem domain will be mixed up with the solution domain, especially, if the usage of micro services is not transparent. Thus, the architecture will very likely be technology-based instead of problem-based. Note, that this is another variant of Conway's Law: "show me your architecture and I will know the technologies it depends on".</li><br /><li><b>External Architecture Quality</b>: As we all have learned the painful way, main reason for architecture failure typically are not functionality aspects. Main reason are quality attributes like performance, extensibility, fault tolerance, and so forth. For each of these quality attributes, well known design strategies and design tactics are available that present alternative solutions how to introduce the respective quality into the architecture. Architecture design uses utility trees and scenario diagrams to specify external quality requirements. These quality attributes are often crosscutting, which is why complex networks of services make it hard and sometimes even impossible to design and implement quality attributes. </li><br /><li><b>Infrastructure</b>: We may use a technology stack for leveraging the micro services architecture pattern or we may build it ourselves. The latter option is a no go, because we are not in the middleware business, at least most of us aren't.</li><br /></ul><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-27523326933370662642012-07-22T12:46:00.001+02:002012-07-22T12:49:36.843+02:00Reviewing systems - the sum is more than its parts<p>The profession of a software or system architect is not only about creating new systems, it is also about assessing and improving existing systems. If change is the rule and not the exception, architectures grow and change continuously. To keep them sustainable, review and assessment techniques are of uttermost importance.</p> <p>An architecture review consists of different phases:</p> <ul> <li>In the <em>clarification or scoping phase</em> we need to define the goal of the architecture  review as well as the one to three key questions the review is supposed to answer. For example, a goal could be to validate that our new system architecture implements the desired dependability quality attributes appropriately. One of the key questions could be: “can the software architecture achieve five nines of availability?”</li> <li>In the<em> analysis or information phase</em> we need to read documents, interview stakeholders, check code and test cases, watch demos, among other things, so that we are able to answer the key questions.</li> <li>In the <em>evaluation phase</em>, reviewers investigate the strengths, weaknesses, opportunities and threats imposed by the current system and related to the review goal. If there are risks in the system, reviewers should define recommendations to mitigate these risks.</li> <li>In the <em>feedback phase</em>, all information (findings, recommendations) provided by the review is returned back to the team responsible for system development.    </li> </ul> <p>These phases roughly correspond to the model specified by RUP (IBM Rational Unified Process): Inception, Elaboration, Construction, Transition.  </p> <p>Unfortunately, there are many different types of reviews and review techniques, qualitative and quantitative reviews, scenario-based and experience-based reviews, code, design and architecture reviews. So which one should we choose for which purpose?</p> <p>In my experience, it is often best to conduct an experience-based review as outlined  in the description of the phases, but integrate other review techniques as well, for example:</p> <ul> <li>If quality attributes are in the main scope of the review, ATAM (Architecture Tradeoff Analysis Method) could be added to concretize quality attributes using scenarios and compare them with the actual architecture, thus determining the risk themes.</li> <li>For obtaining information from stakeholders on architecture issues ADRs (Active Design Reviews) are a complimentary means instead of relying on interviews only.</li> <li>Quantitative assessments such as metrics, prototypes, simulators help to obtain more detailed information about the system under review and its capabilities and limitations.</li> <li>Code and design reviews help reviewers gain more insights about the details of the system. Of course, the code and design reviews are constrained to the parts relevant for the overall review goal.</li> </ul> <p>The toolbox of the software or system architect should contain the relevant review techniques. Whenever architects are involved in a review, they should determine in the clarification phase which techniques they will use in the subsequent phases. </p> <p>Note, that reviews do not need to follow a waterfall model. You may and should use an agile approach with answering the most relevant key question or its most important aspects first using time-boxed increments.  </p> <p>Such reviews may require weeks, but can also be conducted as flash reviews on one day. The more you are able to narrow the review scope, the less effort it takes. If you conduct regular reviews after each increment, the architectural deltas are typically small which lets you narrow the scope of your regular reviews which can then be done in a day. The findings are used to refactor and improve the system architecture.</p> <p>Reviewers should be experienced in software/system architecture as well as in review techniques. For building up a review culture in your organization, start enabling the lead architect to conduct reviews. Then, use the master-apprentice model to constantly increase the review skills in the rest of the team. </p> <p>But as mentioned earlier: do not rely on one single review method but establish a toolbox of review and assessment techniques that can be used in combination to enforce architecture sustainability.</p> Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com6tag:blogger.com,1999:blog-5119025.post-73606866968840296352012-01-07T14:45:00.001+01:002012-01-07T14:45:59.555+01:00TrappedDSLs are currently being promoted by a large number of activists. If you ever read the seminal Pragmatic Programmers book, you'll also find such a recommendation there. So, will the software engineering universe soon turn into a paradise?
Let me tell you two examples from industrial practice:
Once upon a time, in the logistics domain, a smart and lazy software engineer introduced a new language just for himself, in a very ad-hoc manner. When they saw the result, all his colleagues got very excited about the concept and started extending the language for their own purposes. The language inevitably grew and grew. Soon, an increasing number of system parts were depending on the language. Unortunately, when our smart inventor reached a high age, he had to leave and let his colleagues alone. But now, there was no one left in the company who really understood the language concept and its realization. It is claimed that the language keeps still growing. Other theories assume the system had to be completely rewritten in the meantime.
In another universe and another time, another software engineer was convincing his fellows to introduce DSLs. All the IT dwarfs went crazy inventing their own languages. Yes, they even made competitions who could come up with the most fasinating or useful language. After a while, the DWARF software system was nothing but an ocean of languages. Only wizards were able to generate a working solution. When a furious dragon (= customer) attacked their town, everything imploded. No one has a clue where the remains of the dwarf town are located.
What can we learn from these failures?
DSL design is (like) architecture design. An ad-hoc language will lead to design erosion, and cause more harm than good in the long run.
DSLs represent strategic design tools that should only be in the hands of experienced designers.
There are purposes for which DSLs are a perfect fit. But there are also circumstances where they shouldn't be applied. Overdoing it increases accidental complexity.
It is like component-based re-use. Each DSL should only have one responsibility.
Plan to grow the DSLs systematically, refactor it and verify its correctness. Communicate with all stakeholders that are affected by the DSL.
Consider the design choices for the language syntax. Should it be a text-based language, a graphical language, or both?
Mind the XML hell - long, long ago almost everyone was striving for XML based languages. But in some <irony> rare </irony> contexts XML made writing configurations and documents ineffective, uncomprehensible, and tedious. I've seen systems wirh thousands of XML files.
Let experts build and grow languages.
It is amazing how (often) our discipline gets addicted to all kinds of panacea. Once you are lost in the buzzword jungle, all systems and stakeholders will suffer fom DSL paranoia.
DSLs are an awesome means for boosting productivity. However, used by the wrong persons or with the wrong motivations, it is easy to shoot yourself in the foot.
<br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com5tag:blogger.com,1999:blog-5119025.post-5236991225731541062011-10-12T23:22:00.001+02:002011-10-12T23:22:16.184+02:00Architecture GovernanceIn a large project a product line is developed that supports multiple smart phones. The platform development team puts all code in its super magical configuration management system. Whenever one of the phone teams starts with a product development, it extracts the platform code from the CM system, and adapts it to its own needs. With other words, even the core assets (i.e., the platform itself) are modified. Of course, the platform team is not aware of all these activities. C'est la vie.<br /><br />This is an example for reuse, albeit one which rather serves as a war story. How come? Imagine, the next version of the mobile platform is going to be developed. How can the domain engineers ever evolve a system that was modified by various product development groups in different ways? The product line has been literally transformed in a bunch of one-off applications by unsystematic reuse. One approach to prevent such architectural war stories is what we call Architecture Governance. <br /><br />Architecture Governance is a systematic approach for managing architectures and controlling all modifications in order to ensure quality and sustainability. This holds for all modifications, those for developing the system and those for evolving it. <br /><br />But who is in control? On one hand, there should be an architecture control board in charge of the technical architecture, and on the other hand, another control board should be in charge of the business and business strategy - Luke Hohmann once coined the terms "tarchitecture" and "marchitecture" in this context.<br /><br />It s important to think about governance in general. Architecture governance is no island. It must be balanced with IT Governance, SOA Governance, and other classes of governance. Governance is about preferring control and monitoring over trust.<br /><br />WTF does "control" mean? In the product line example, it means we need to introduce an owner of the platform. And we need someone responsible for business & strategy. Only these two "persons" are allowed to decide about modifications of the platform in terms of business and architectural sustainability. Of course, the business control board is supervising the architecture control board. Eventually, it is the business that drives the development.<br /><br />For architecture control, guiding principles should provide policies for modification and evolution. Obviously, the internal and external quality of the system as well as its expected behavior must never be compromised. That's why we need tools to check and enforce policies, tools to assess the architecture, and test suites to obtain respective information. This is where monitoring comes in. By the way, "tools" in this context also include reviews. And mind the gardening activities! How can a system be systematically prepared for modification. This is where reengineering and in particular refactoring become important.<br /><br />But there are even more details to bother about:<br /><br />We need to address legal and regulatory issues. Any change must not violate such standards. Think of safety features for medical products as an example.<br /><br />It is important to care about quality of service. Assume, a modification leads to a breakdown of KPIs or SLAs.<br /><br />Don't ignore or neglect quality attributes in general. Does a modification influence sensitivity or tradeoff points? Will it introduce new risks?<br /><br />An issue might also be patent scanning. Is there any new code introduced such as an Open Source Software that violates intellectual property rights?<br /><br /><br />But it is not only about tools. It is also about a governance process as well as assignment of responsibilities to roles and persons. For example: Who is allowed to change, configure, update, or add what and when. What measures are necessary in order to guarantee quality and sustainability. How does information flow between the different actors. What happens in the case of policy violations. <br /><br />Note: This does not only hold for product lines but also for one-off applications. Unsystematic modification almost always leads to potential problems, because of unwanted side affects, accidental complexity, and lack of transparency.<br /><br />What does this imply for the design of new architectures? It is the same issue: we need a business owner, an architecture owner, a design and implementation strategy, test-driven development, refactoring, and so forth. I happen to have introduced all these topics already in my blog :-) Modifying and evolving a system is only a special case of designing it. <br /><br />By the way: An agile development process supports architecture governance in that its iterative incremental approach already introduces control and monitoring points. <br /><br />In practice, there are many ways to deal architecturally with architecture governance. I cannot describe all of them for the sake of brevity. But let me give you some examples:<br /><br />Using the Layers patterns in a strict sense helps to protect subsystems. Think about Safety segregation.<br /><br />Clean Coding represents a good way to make control and monitoring easier.<br /><br />Well documented software architectures are easier to modify and control.<br /><br />There are many architectural means to foster governance. <br /><br />Keep in mind that architecture governance is not about controlling architects or developers. Instead, governance helps architects and developers to keep in control. It requires additional efforts but its RoI is very high. Ignoring governance leads to project failure, overcomplex systems, buggy implementations, and design erosion. And eventually to dissatisfied customers and users (and dissatisfied architects and developers). Don't let the software govern you. Govern the software!<br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com5tag:blogger.com,1999:blog-5119025.post-87167387796962881692011-10-06T23:02:00.001+02:002011-10-06T23:03:50.573+02:00Steve Jobs, 1955 - 2011Born in 63, I had the great opportunity to grow up with personal computers. Even around the Eighties Woz and Steve Jobs were already legends for their visionary minds and creations. I liked the first LISAs and Macs but unfortunately could't afford to buy one. <br /><br />When Steve had to leave Apple in 1985 he created the NeXT which has been an extraordinarily successful machine, not so much from a business perspective but from a creative one.<br /><br />But his popularity exploded eventually after Steve Jobs rejoined Apple as CEO. Steve Jobs stands for creativity, vision, courage, leadership, charisma. He honestly lived and worked literally day and night for Apple and its users. In contrast to many competitors he gave products not only functionality but personality and style.<br /><br />What only few people know is that Steve also had a strong influence on software. Not only on UI topics but also on programming languages, operating systems, and many other aspects.<br /><br />I personally have never been an Apple "fan boy" before I bought my first iPod, but now I love their products. However, I must confess that other mothers also have beautiful daughters. Nonetheless it is evident that Apple has always been the driver of amazing innovations that were frequently copied by competitors.<br /><br />That's the reason, why I want to thank you, Steve. Many things we take for granted simply wouldn't exist without you. I fell deep respect for you and your work. I am sure, you'll always be unforgotten as one of the leading personalities in the IT Hall of Fame. And I really hope, your spirit will always persist within Apple. <br /><br />The last thing I learned from you, although this has been a sad and bitter experience is: Carpe Diem! And that we should see the people behind and in front of all these nice products.<br /><br />Send my greetings also to Doug Adams. So long, and thank you for all the apples!<br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-52428329548495562892011-09-20T17:38:00.001+02:002011-09-20T17:39:38.966+02:00Using War StoriesFor the education of software architects we are using war stories to emphasize important learnings. The whole curriculum is based on the mantra of learning from failure. Errare humanum est! Thus, it is important to see what can go wrong and how to deal with it in a better way. Everyone of us architects knows a whole bunch of war stories from the own career. I also caused failures, but learned from them. It is basically the same like children when they learn to walk. They'll fail but keep on trying until they eventually succeed. It is not a shame to think about own failures. Some cultures force people to always appear perfect which is a bad fundament. As we all encounter failure, it is better to learn what exactly went wrong and why instead of just hiding it from others and ourselves. When we educate architects we do not just teach them theory but also and even more practice. As Einstein once said, "in theory, theory and practice are the same. In practice they aren't."<br /><br /><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-64929083827321953302011-09-16T22:13:00.001+02:002011-09-16T22:13:45.189+02:00Fractal Design<p>If you are going to design a system, you basically need to identify prioritized main use cases and quality attribute scenarios among other forces to design the system. Together with a problem domain model that shows the inner constituents, and a context model that integrates the system into its environment, the design can evolve in a evolutionary way. As a result of the design activities, architects are able to introduce subsystems as well as their relations according to functional or qualitative drivers. And, of course, interfaces start to appear, each of them defining one specific and explicit role of the subsystem.</p> <p>A subsystem is itself integrated into an environment – the enclosing system under development. So the subsystem can also act as a system. Thus the same principles apply for the subsystem acting as a system itself – you may even define use cases and quality scenarios for the subsystem with use cases and scenarios being derived as a subset of the “outer” use cases and scenarios. After this step, “subsubsystems” are created, often coined “components”.</p> <p>What we do here is applying the same design principle in a top-down manner. </p> <p>But is it useful or possible to apply the principle infinitely? No, because at some level the solution domain is shining through. Solution domains tend to introduce their own composition techniques such as assemblies, bundles, EJBs, services, classes and objects. If the top-down design approach reaches this level, designers must map the architecture artifacts to the solution domain. Maybe, we should call this level architectural twilight zone or the problem-solution-boundary <img style="border-bottom-style: none; border-left-style: none; border-top-style: none; border-right-style: none" class="wlEmoticon wlEmoticon-smile" alt="Smile" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBe43Q3uG6FzRfGZDGLa89pbTfUjNV7GoJJlDQFkPCjNSfq_wnJB8DxB_vDVvJNWHTKUNYxF2QV76DyKUmBNklIDN0Vpuo_iyLIG0JRdQ8gVsxMo3W-bqhhUuLAUX-zM0Yjqve_Q/?imgmax=800" /></p> <blockquote> <p><font color="#333333" size="2">Side remark: To overcome the twilight zone, we could introduce DSLs and use Model-Driven Software Development.</font></p> </blockquote> <p>As a rule of thumb, we typically obtain 2 sublevels (subsystems, subsubsystems) as architects. If less, the design is too abstract and vague. If more, we are introducing too many details.</p> <p>One of the core challenges in this context is the fact that there might be different strategies to view a domain and thus different ways to cut a system into subsystems. </p> <p>Partitioning a system into subsystems independent of the hierarchy level is influenced by functional aspects and the problem domain. Thus, subsystems should introduce meaningful subdomains of the surrounding problem domain. With other words: methods like Domain-Driven Design together with some extensions can help nicely. </p> <p>No matter what you do, there will always be crosscutting qualities and topics. The same observation can be made when structuring an organization into divisions, departments, groups. Have you ever seen an organization without overlapping units? The introduction of crosscutting concerns may introduce new sub^<sup>n</sup>systems, add new interfaces or even change their implementation depending on the invasiveness of the concrete concern. Each concern can be considered a subordinate view mixed into the subsystem respectively its domain.</p> <p>Architecture design is basically fractal design up to two levels of depth. The priorities of use cases and quality scenarios as well as their properties (strategic versus tactical) define how and in which order the functional model needs to be refined hierarchically by integrating scenario-based views. </p> <p>Of course, one person’s solution domain can be another person’s problem domain which is why exceptions to the aforementioned rule might apply.</p> Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com2tag:blogger.com,1999:blog-5119025.post-53025910284533667012011-09-08T12:36:00.001+02:002011-09-08T12:36:38.442+02:00The Telephone Test for Software ArchitectureWe all know that a software architecture should reveal two properties among many others for an adequate internal quality:<br /><br />Simplicity implies that the software architecture only addresses inherent complexity without introducing accidental complexity. Since, there are typically several ways to solve a problem, there is no simplest architecture available. Instead, there rather are solution architectures that follow one specific solution path with a minimal number of artifacts. As some quotes propose, simplicity is achieved if you cannot take something away from your system without failing to meet its specification.<br /><br />Expressiveness implies that the artifacts of your architecture are easy to understand. That is, artifacts should have expressive names, and each responsibility should be assigned to one artifact. Thus, components with a multitude of responsibilities are often a bad idea such as are responsibilities spread across multiple components. However, it is particularly difficult to achieve the latter goal due to cross-cutting concerns. An additional step to achieve expressiveness is having role-based, explicit interfaces with concrete contracts.<br /><br />But how can you test simplicity and expressiveness? There is a good low-tech suggestion for doing this: Let a software architect explain the architecture to an engineer not involved in the project, for instance using a phone call. Limit the time to - let's say - 10 minutes. If the other engineer gets a good idea of the architecture, it is an indication that the architecture is simple and expressive. Of course, I am assuming that the software architect is a person good in communication as I expect from architects, anyway. <br /><br />Some might argue that design metrics could also help in this context. Indeed, metrics provide some insights. But we shouldn't forget that metrics analyze the structure, not the semantics. Thus, they are not capable of deciding about expressiveness.<br /><p class='blogpress_location'>Location:<a href='http://maps.google.com/maps?q=Eduard-Schmid-Stra%C3%9Fe,Munich,Germany%4048.127691%2C11.578741&z=10'>Eduard-Schmid-Straße,Munich,Germany</a></p>Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com5tag:blogger.com,1999:blog-5119025.post-19528886057373000382011-09-07T13:39:00.001+02:002011-09-07T13:39:39.490+02:00Apples and Oranges<p>How often do we hear statements like how immature software engineering is compared to other engineering disciplines? Of course, construction building is a very old craft where engineers could collect huge amounts of knowledge, methods, and experiences over thousands of years. </p> <p>On the other hand, there is a huge gap between traditional engineering disciplines and software engineering:</p> <ul> <li>While other engineering disciplines are focused on specific domains, software engineering is supposed to support countless problem domains.</li> </ul> <p>This is why we came up with technologies that are more general in terms of problem domains such as:</p> <ul> <li>UML</li> <li>Architecture Definition and Specification Languages</li> <li>Generic Design Patterns (GoF)</li> <li>Components and Services</li> </ul> <p>However, there is a huge trap using such approaches. Due to their general and generic nature, they are far away from the realities of the problem domain. Thus, communication between software engineers and non IT-knowledgeable stakeholders does suffer.</p> <p>It is interesting how many experts still emphasize those general-purpose tools and technologies. So to speak, they are addicted to the One-Size-Fits-All drug.</p> <p>For instance, look at architecture description languages that introduce general components and connections. The problem is that for a concrete problem domain, such generality simply does not work. It is a difference, whether you are dealing with a medical imaging modality or a VoIP platform. Yes, we can … call all building blocks of such systems components or subsystems. And, yes we can … call all interaction paths connections. But, here we are unifying different concepts under one common umbrella.  </p> <p>What software people often forget is that there are two universes:</p> <ul> <li>the Solution Domain with all its supporting tools and technologies. In this domain, general-purpose approaches as the aforementioned make sense,</li> <li>AND the Problem Domain.</li> </ul> <p>In the Problem Domain we can not naively leverage concepts like UML, Components or General Design Patterns. Instead, we need to follow a Problem Domain First Approach. Thus, it is necessary to start with concepts of the problem domain. What does that mean in practice?</p> <ul> <li>Introduce DSLs and Domain-Driven-Design to cover the problem domain instead of relying on UML. As a matter of fact we can use the underlying meta-model of UML as a base. It is possible to evolve such a language in an iterative approach.</li> <li>Think about the basic building blocks not as components and subsystems, but use the terminology of the problem domain. You can define your own problem-specific components, subsystems, or services for this purpose. This is exactly what we did for a very large Enterprise Communications System.</li> <li>Consider the availability of Analysis Patterns for your problem domain and subdomains. Use them if available. </li> </ul> <p>Building the Software Architecture then consists of mapping the problem domain to the solution domain. How could we do that? I mentioned the Onion Model several times. So steps could be:</p> <ul> <li>Understand the problem domain and build a model of it jointly with domain experts</li> <li>Leverage use cases to understand what the system under development is supposed to deliver from a black-box view.</li> <li>Use the problem domain model to map the use cases to the problem domain artifacts introducing further functional aspects.</li> <li>Stepwise extend the architecture by using strategic quality-attribute scenarios with descending priorities.</li> <li>Prepare the architecture for tactical requirements by using the tactical quality attribute scenarios with descending priorities.</li> <li>Do all this just considering three abstraction levels to limit complexity.</li> <li>Apply architecture patterns to structure the overall system.</li> </ul> <p>Mind the Change! Unlike in many other disciplines, software is considered so soft that it should even support strategic late changes. This is like switching your project goal from creating a coffee machine to creating a power plant. Unfortunately, customers are rarely aware of this problem because they are being constantly told by sales and marketing that changes are no problem. Thus, make customers and maybe even more sales and marketing aware of this fact. And use an agile approach to embrace change. But also have courage to deny changes. This is another reason why the Problem-Domain-First approach is so important. How else could you know what changes or extensions are typical for a problem domain? Think of the tax legislation in your country. It is not helpful to only think of interception and extension hooks or Strategy pattern instantiations in this context. </p> <p>All general approaches for software engineering have their value and can be used as the underpinnings of your systems. But they should not be used for covering the whole software development from problem domain to solution domain. This is also what we can substantially learn from other engineering disciplines.</p> <p>Thus, don’t mix apples and oranges. Otherwise, the result may disappoint you.</p> Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-74280648526579443282011-09-04T13:36:00.001+02:002011-09-04T13:40:32.880+02:00EmergenceAs explained in my last posting, emergence is a powerful approach for solving some hard problems in a non-deterministic way. It's a kind of magic? Not really, it is more an architecture pattern.<br /><br />What do we need for implementing emergence?<br /><br />We require <br /><br />a set of active agents,<br />a (set of) communication mechanism(s),<br />a common goal the agents implicitly or explicitly share,<br />an environment,<br />cooperation strategies of agent (optionally),<br />a method to determine when to stop (optionally).<br /><br />The agents are active in that they execute some functionality in order to reach a goal or subgoal. They communicate with other agents to transfer information. And they recognize when their work is done or when they should stop.<br /><br />There is a lot of possible variation here:<br /><br />Agents might all share the same role (behavior) or they may have different roles.<br />Agents might be organized in a peer-to-peer fashion or hierarchically.<br />Agents might share the same goal or have individual or role-based goals.<br />Control might be distributed across all agents, or only be assigned to one or a multiple of them. <br />Agents might be preinstantiated at start-up or adapt their number or roles according to environmental conditions or achievement (being born, living, dying).<br />Agents might be able to give life to new agents. They may also be able to terminate other agents, even themselves.<br />There even could be some environmental conditions that result in death or birth of agents.<br />Communication might be one-to-one or one-to-many or abide to any other message exchange pattern.<br />Communication might be synchronous or asychronous.<br />Agents might adapt their behavior including communication due to environmental changes or interaction with other agents.<br />Which implies, the environment might change.<br />Agents might "move" in their environment.<br />There could also be diverse neighbor environments.<br /><br />In a Map-Reduce system we have a controlling agent and subordinate agents. The controlling master might instantiate slaves depending on problem complexity. Goal of the system is to reach a common goal such as finding all specific entities in an environment. The master communicates with the slaves and provides a subgoal to each of them. The slaves solve the subgoal and may act as masters for their subgoal, thus recursively applying the pattern. Slaves send their findings back to their master which then recombines partial results to the common result.<br /><br />In Leader-Followers, the leader is always the controlling agent. When it receives an information from its environment, it promotes a follower to become the leader, and switches its role to worker. A worker who is done with a job, switches its role into follower and enters the followers list.<br /><br />All these systems reveal emergent behavior. And this also resembles swarms like ant populations. There are different roles such as queen, warrior, worker, etc. The population's goal is to sustain by ensuring the survival of the population and their offspring as well as giving birth to new populations. When searching and locating food sources, ants use pheromons for communication. They show warrior strategies for fighting enemies. And they always adapt to their environment in terms of food, war, weather conditions. Ants might even adapt distribution of their roles such as creating more warriors when necessary. The whole population appears as a kind of smart creature.<br /><br />A human might also be considered as a whole consisting of smaller agents (cells) and this is how evolution actually developed complex life forms.<br /><br />And if you think about it, the Web itself is just an emergent system.<br /><br />When we are going to build such an emergent system, we need to consider all these aspects defined above such as roles, hierarchies, communication styles, goals, controls, environment, adaptation, cooperation strategies.<br /><br />One promising way to implement such systems are Actor-based approaches.<br /><br />If we had a toolkit for emergent system that allows to configure the different variation points, we could play around with the concept. Akka is one of the excellent frameworks that could serve as a base for such a toolkit.<br /><br />So, eventually emergent systems are developed using emergent design approaches. Until then, there is a large journey ahead of us.<br /><br /><br /><br /><br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-27066530672553730312011-08-31T12:47:00.001+02:002011-08-31T12:59:42.188+02:00The Whole is more than the Sum of its Parts – Creating Complex Systems from Simple Entities<p>The Internet represents an ubiquitous infrastructure that enables complex systems such as clouds, SOA-based systems, e-mail, or Web 2.0 applications. Despite of its overall complexity, the basic ingredients of the Internet are rather "simple" and easy to understand. Think of parts like TCP/IP, HTTP(S), HTML, URIs, DNS, IMAP, SMTP, REST, or RSS, to name just a few. </p> <p>Another example though not software-related is Lego. Children (and adults :-) can build the most sophisticated systems from very simple Lego bricks. And even in biological systems like ant populations or in physical systems - like the whole universe itself - we can make similar observations. </p> <p>Infrastructures such as the Internet with high inherent complexity consist of such simple constituents. Why are we still capable of providing new kinds of innovative software applications based on technologies that are more than 20 to 40 years old? What do all of these examples have in common? </p> <p>Well, they basically combine simple building blocks with a set of interaction-, adaptation-, and composition-mechanisms. Moreover, they allow to compose higher level building blocks (from simpler parts) that support their own abstract set of interaction, adaptation, and composition. Think of Layered Architecture! And mind the fact that the building blocks in this context are not only static entities but (might) also include behavioral aspects making them active objects. </p> <p>Interestingly, the composition or interaction of parts leads to cross-cutting functionality that wouldn't be possible otherwise. Most recently, there have been scientific papers claiming that the human personality is "just" a result of cross-cutting behavior - but that what I'd like to mention as an interesting side-remark.</p> <p>In general, approaches are summarized under emergent behavior or emergence when we speak about (inter)active parts. In this context, I am considering the term "emergent" as a broader concept also including passive ingredients. </p> <p>But why should we care? What can we software engineers learn from the principle of emergence, evolution or composition? </p> <p>It is possible to build very complex systems based on simple building blocks. That is very obvious because eventually all physical systems are composed from simple elements. However, we need to address the challenge, how, why and when to apply such composition techniques to our own software. And we also need to address the issue of quality. </p> <p>While it is theoretically possible to build every system using very small, atomic parts, this would lead to unacceptable development efforts, bad maintenance and low expressiveness. Think about using a Turing Machine for implementing a word processor. Hence, the gap between the behavior we are going to provide in resulting applications respectively artifacts and the basic building blocks used for their construction shouldn't be too wide. </p> <p>On the other hand, we’d like to support the creation of a whole family of applications in a given domain. (If you are knowledgeable about Product Line Engineering, this should sound familiar.)</p> <p>There are three fundamental possibilities to construct such systems:</p> <ul> <li>Bottom Up, i.e., creating a base of entities that spans a whole domain. We can view the domain as a “language” with the base acting as a kind of “grammar”. </li> <li>Top-Down: i.e., thinking about what kind of artifacts we’d like to create and trying to figure out a set a set of entities from which these artifacts can be composed via one or more abstraction layers. </li> <li>Hybrid: maybe, the most pragmatic approach which combines Top-Down and Bottom-Up. </li> </ul> <p>To make things even more complex, the “grammar” could be subject to evolution which might change the “grammar” and/or the language.</p> <p>Very theoretical, so far. Agreed, so let me show a practical show case.</p> <p>A prominent example is the Leader-Followers pattern. Roughly speaking, an active leader is listening to an event source. Whenever the leader detects an event, it turns into a worker and actively handles the event, but only after it selected one of the followers to be the next leader. As soon as the “new born” worker has completed its event handling activity, it turns into a follower. And then the whole story might repeat again. This pattern works nicely for applications such as a logging server. Its beauty stems from the fact that active agents comprise a self-organizing system, leading to non-deterministic behavior.</p> <p>Engineers often prefer centralized approaches with one or more central  hubs or mediators. That is, we like to introduce a central control. There must be someone or something in control to achieve a common goal, right? </p> <p>In fact, this assumption is wrong. In some cases, it is necessary to have a control, but this control does not need to be a central component but can be distributed across several entities. Let me give you a real life example:</p> <p>Some day, you are visiting a shopping mall. After having a nice time, you are leaving the mall. But you forgot where you left the car. Thus, all members of the family start searching in the parking lot until someone locates the car and notifies everyone else. Normally, at least in some families, this search activity would be rather unorganized. Or each family members would agree to search in a specific area, but no one would prescribe how the search pattern looks like. Nonetheless, all share a common goal and will finally succeed. (Let me just assume, the car hasn’t been stolen :-)</p> <p>An architecture concept that resembles this situation could be active agents that communicate with each other using a blackboard. Just assume, we are building a naive web crawler. On the blackboard you could write down the link where you’d like to start the web crawling. An agent takes the token (i.e., the link) and starts to search the Web page. On the Web page it finds further links which it writes to the blackboard, each link representing a separate token. Now, all active agents can access the blackboard, grab a token, analyze the corresponding page and write the links they find on the blackboard. But, wait a moment, when does the search end? The problem boils down to the question how we handle duplicates (Web sites the system already parsed). For this purpose, we could introduce a hash table where we store pages the agents have already visited. If an agent reads a token which represents an already visited URL, it just throws the token away and tries to grab another one. As soon as there are no more tokens available, the crawling is completed. </p> <p>Sometimes, we also need (single or multiple) controls. For instance, in most Peer-to-Peer networks, the search for a resource starts using an initial set of controls that aggregate information about their environment. But the search or the registration of resources are conducted in a self-organizing and decentralized way. Note, that the introduction of controls or hubs is not necessary for functional reasons, but helps to increase scalability and performance.</p> <p>A Sensor (or Robot) Network is an example, where we might even need no control at all. Just consider the case, that we distribute sensors across a whole area to measure environmental data. Even if some of these sensors fail, we are still able to gather sufficient data, given a certain amount of sensors is available. But, of course, one could argue, that there must be someone central who monitors al the sensors. By the way, the network of weather stations is an excellent show case for this strategy.</p> <p>To summarize this part, in such scenarios with active components, we can have the full range of centralized control, decentralized control, or no control at all. Anyway, the important thing is that each active entity has a predefined goal and there is communication between these entities either using a Peer-to-Peer communication style or one (or more) information hub(s).</p> <p>As already mentioned, we could add evolutionary elements by letting the agents adapt to changing environments using strategies like survival of the fittest and uncontrolled modification. Genetic algorithms are the best example for this architectural style. Like in neuronal networks, the main challenge for a software engineer is the fact that such systems mostly reveal their functionality using cross cutting and non-deterministic behavior. This makes it difficult to create such systems in a top-down way. Most of the time, such systems are composed bottom-up leveraging a “configurable” evaluation function. With the evaluation function, systems can measure how good the results are and adapt dynamically until the results reach a required level of quality.</p> <p>Now, I’d like to move to more “conventional” software architectures. As we know, all software architectures are idiomatic, usually providing a stack of idiomatic layers and composition of idiomatic subsystems  and components. (Whatever, “subsystem” and “component” mean in this context :-) Note, that this also holds for patterns and especially for pattern languages.</p> <p>By  “idiomatic” I am referring to the point that architectural artifacts offer a kind of API which define syntax and semantics from their consumers’ perspective. By composing these artifacts, higher level functionality can be addressed using lower level functionality. Artifacts at the same level cooperate to achieve composite behavior.  Again, we are introducing entities, composition, and communication. Cross cutting behavior addresses composites that reveal specific functionalities or quality attributes. To this end, even quality attributes could be considered functional: they may imply additional functionality or comprise cross cutting behavior.</p> <p>With other words, in software engineering we face the challenge of hierarchies and composition  respectively integration of languages. </p> <ul> <li>The more complex these languages are, the more complex it is to use them. </li> <li>The more complex the compositions of languages are, the more complex it is to integrate them into a whole. </li> </ul> <p>Thus, we need to balance between complexity of these languages and the complexity of their composition. This balance of forces can be achieved by building an appropriate hierarchy of he languages.</p> <p>Thus, yet another definition of “Software Architecture” could be:</p> <blockquote> <p>Software architecture is about the hierarchical integration of artifacts that balances the idiomatic complexity of its constituents and the complexity of their composition. In order to meet the given forces, it addresses strategic design decisions using different viewpoints, prepares mandatory tactical aspects, and abides to common guiding principles.</p> </blockquote> <p>The secret of good design is appropriate partitioning in core parts which are then composed to build a whole such that it can easily address inherent complexity, without introducing accidental complexity. In a platform or product line, this can be really challenging, because we must enable application engineering to create each member of the product family by using the common parts, while allowing to deal with variability. As we are dealing with hierarchies of artifacts, this turns out to be even more complex. When extending the scope of a product line, there might be implications on all aspects of the reference architecture.</p> <p>The goal of architecture design should be to design complex functionality by composing simple artifacts in a simple way. The tradeoff is between complexity of artifacts and complexity of composition as well as finding an appropriate hierarchical composition. All approaches such as AOP, model-driven design, Component-Based Development, or SOA try to achieve this nirvana. However, introducing basic artifacts and composition techniques is beneficial not sufficient. Instead, it is crucial to cover the problem domain and its mapping to the solution domain.</p> <p>This is what all the well-known definitions of Software Architecture typically forget to mention :-)  </p> Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1tag:blogger.com,1999:blog-5119025.post-61072662498926573172011-08-25T16:29:00.001+02:002011-08-25T16:29:22.021+02:00Make it explicitIn development projects one of the main challenges comprises clarifying the requirements and assumptions. Knowledge about the problem (and solution) domains proves to be critical in this context. How else would a software engineer understand the requirements? I claim that one of the most critical problems is implicit knowledge which is knowledge taken for granted by some stakeholders, e.g., the customers, but not known to others. <br /><br />An appropriate approach to address at least parts of this issue is to conduct a KANO analysis where engineers will define features all customers would expect, but also excitement features that may surprise the users in a positive way. Take the iPhone as an example: Expected features include the ability to make phone calls, to maintain a contact list, or to send SMS messages. If a product lacks one of these, customers will be disappointed, but even the coverage of all expected features won't make a customer excited. Excitement features of the iPhone include the Touch Screen and the possibility to add functionality using apps. It is important to mention that today's excitement features will become expected features in the future. Obviously, developers of a mobile phone need to know the expected features in order to prevent any disappointment. However, in many cases these expected features are only known implicitly which works fine if you are familiar with the problem domain, but leads to trouble if this is not the case. Thus, the first conclusion is to make these requirements explicit. The introduction of a problem domain model and of a feature tree prove to be excellent ways to understand the domain and the features as well as their dependence and interaction. <br /><br />So far I mentioned functional aspects, but implicit assumptions about quality attributes should not be neglected either. Suppose, it would take minutes to find a person in the contact list or to initiate a phone call. Such cross-cutting concerns may not appear in the requirements specification, but nonetheless are essential for successful development. Thus, also the quality attributes must be made explicit which does not only turn out to be important for design but also for the establishment of a test strategy and test plans. <br /><br />Besides requirements the business strategy and business case often make implicit assumptions such as the kinds of users or market segments or features or future evolution. Whenever architects are not informed about these assumptions, they can neither check their feasibility or validity nor can they provide the right solution. How could someone provide a solution for a vaguely understood problem anyway? Conclusion: always make sure that you obtain sufficient information from management. And mind the ambiguity problem! For instance, I once asked several product managers about their usability requirement. It turned out that every single one of them had a completely different understanding of usability. This is the reason, why a common model and effective communication are so important.<br /><br />Another problem area is the mapping from the problem domain to the solution domain. Developers may use particular technologies, because they find these the right ones for addressing the problem. Or managers may prescribe technologies without knowing their implications. Architects may enforce decisions without giving a rationale. And developers may make implicit but wrong assumptions about the architecture.Thus, making decisions based on explicit assumptions is inevitable for successful design. For example, architects and developers should verify assumptions and document decisions explicitly. If you don't believe this, think about gifts you gave other persons based on your assumptions.<br /><br />A benefit of explicit information is that it can be cross-checked and allows traceability. In a development project all the groups of stakeholders should not consider themselves as disconnected islands. Instead, they should share knowledge by using effective communication strategies. Also, they should make their decision process and decisions transparent to the other groups. Basically, it is like having one common information repository where stakeholders may have different views. But these different views must map consistently to each other. This is the reason why architects are the right persons to enforce making everything explicit. In particular, the development of platforms and product lines is very sensitive to implicit information, because failure to consider this information has an impact on many applications.<br /><br /><br />- Posted using BlogPress from my iPad<br />Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com0tag:blogger.com,1999:blog-5119025.post-68106066655021906692011-04-21T12:25:00.001+02:002011-04-21T12:25:07.411+02:00SEACON 2011 Architecture Day in Hamburg<p>This year the German event <a href="http://www.sigs-datacom.de/index.php?id=seacon2011">SEACON 2011</a> is taking place from 27th to 29th June. The venue is the Atlantik-Hotel in Hamburg, a very nice location famous for its permanent guest, German Pop/Rock Tycoon Udo Lindenberg.</p> <p>I was responsible to organize the architecture day on 29th June. And we managed to get excellent speakers covering a lot of interesting  architecture topics. Jim Webber from Thoughtworks is going to give the keynote address: “Lessons learned in large HTTP-centric Systems”. Attendees will also have the opportunity to visit half-day tutorials on Dynamic Software Analysis (Johannes Siedersleben) and Software Quality (Erik Dörnenburg). In addition, the <a href="http://www.sigs-datacom.de/seacon2011/konferenz/konferenzprogramm.html">agenda</a> is filled with excellent talks by renowned speakers and practitioners.</p> <p>For software architects this will be a highlight in 2011. The event SEACON Architecture Day might become an established event on Software Architecture. It is up to you. Attend and enjoy!</p> Michaelhttp://www.blogger.com/profile/11984599832785431031noreply@blogger.com1