tag:blogger.com,1999:blog-7826786244968404549.post3650542260972668944..comments2023-09-05T23:58:35.122-07:00Comments on Simple Architectures for Complex Enterprises: Attacking Architectural ComplexityRoger Sessionshttp://www.blogger.com/profile/16946430426943308823noreply@blogger.comBlogger12125tag:blogger.com,1999:blog-7826786244968404549.post-38742378805282127152009-10-06T08:41:49.460-07:002009-10-06T08:41:49.460-07:00Yeah, agreed: in essence what we're dealing wi...Yeah, agreed: in essence what we're dealing with is a clash of terminology.<br /><br />Your usage is appropriate (or rather, a common usage) for IT-systems.<br /><br />For those of us who deal in systems in the broader sense, 'complexity' has a broader meaning, as described above.<br /><br />In effect, your usage applies to quite a small subset of 'complexity' in the Cynefin etc sense, in which there's an assumption that the 'world' being worked on conforms solely to Aristotelian true/false logic, and in which the concept of 'state' and suchlike are still meaningful. In Cynefin, your version of 'complex' is 'knowable', and hence belongs in the 'complicated' domain, not the 'complex' domain. Hence the confusion here.<br /><br />Which is fine - except that there is a real danger of a '<a href="http://weblog.tomgraves.org/index.php/2009/08/19/term-hijack/" rel="nofollow">term-hijack</a>', where this subset purports to be the whole, preventing necessary usage of the term in its true broader sense. Hence 'twould be useful to perhaps be a bit more explicit that you're only applying this narrow usage ("the number of states in a system"), and not purporting to present "an all-encompassing theory of complexity"? :-)<br /><br />Best wishes, anyway.Tom Ghttps://www.blogger.com/profile/13526969814702899986noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-64005126144131165752009-10-06T04:31:52.508-07:002009-10-06T04:31:52.508-07:00Tom G: Keep in mind that SIP is intended only to d...Tom G: Keep in mind that SIP is intended only to deal with the world of IT systems. While I believe the ideas have more far reaching implications, I have not had the time to explore these implications. For now, I just claim it has value in IT.<br /><br />And I do believe that my definition of complexity ("the number of states in a system") is in line with how many define complexity in related systems. For example, from wikipedia: "In information processing, complexity is a measure of the total number of properties transmitted by an object and detected by an observer. Such a collection of properties is often referred to as a state."<br /><br />I am not trying to put together an all encompassing theory of complexity (at least, not yet). I am dealing with the very real problem of reducing IT failures due to mismanaged complexity. If I can help solve that problem, I will be happy!Roger Sessionshttps://www.blogger.com/profile/16946430426943308823noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-37091946039882780032009-10-05T23:02:20.087-07:002009-10-05T23:02:20.087-07:00@Roger: "I assume that complicatedness has to...@Roger: "I assume that <i>complicatedness</i> has to do with the difficulty to understand a system. I define <i>complexity</i> as the number of states in a system. Complexity and complicatedness are thus related, but still quite different. Consider a bowl containing a single dice. Now add one more dice. By adding one more dice, you have increased the complicatedness of the system by two, but the complexity (the number of states) by six."<br /><br />Note that that "I assume" <i>is</i> only an assumption - a fairly arbitrary one at that. In the Cynefin sense, your 'complicatedness' and 'complexity' would <i>both</i> be forms of complicatedness, because both are 'knowable'.<br /><br />To take your dice example, it's a good illustration of complicatedness - the way in which an additive relationship compounds the level of complication. (And yes, it <i>is</i> easy to use "adds to the complexity" as a shorthand for this - but the danger is that it then leaves us no means to describe true complexity.)<br /><br />So note that you've <i>assumed</i> an additive relationship there: adding one more six-faced die adds six more possible states. In a true complexity, we cannot know beforehand what the number of faces will be, or what the relationship between the dice may be. We can probably identify those <i>at the present time</i> - but they may well change tomorrow. Even in this simple case, the <i>effective</i> number of states will vary according both to knowable and unknowable factors; and the outcome 'states' - the resultant outputs from the relationship will be messier still.<br /><br />For example, what happens when the relationship of the two dice A and B changes at random between A+B, A-B, A*B, A/B, B/A, A XOR B and so on? In many cases we should be able to identify the (nominal) causal-relationship <i>retrospectively</i>, but the key point is that we may not know it at the time. And if our 'complicated' system assumes only an additive relationship between the two dice, it's likely to give us seriously wrong answers when the relationship changes. In true complexity we can only derive the relationship from the answers we get - in other words, we're forced to use non-logical methods (e.g. analogy, metaphor, inductive reasoning) to derive the current underlying logic. If we don't respect the fact that those methods <i>are</i> non-logical, we're again likely to end up in serious trouble.<br /><br />In short, what you've described <i>will</i> work well within an artificially constrained system in which the factors and their relationships (however complicated they may be) do remain stable, do follow a true/false logic, and hence do remain predictable - as is typical for (most of) the internal logic of IT-based systems and systems-of-systems. But this approach will fail as soon as it hits up against any infinity or unpredictability - and the real world is riddled with those. To deal with the real world, we need to respect complexity <i>as</i> complexity - and not pretend that we can 'solve' it all by over-simplistic layers of complication.Tom Ghttps://www.blogger.com/profile/13526969814702899986noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-70680728448107574972009-10-05T06:29:00.563-07:002009-10-05T06:29:00.563-07:00Christopher Alexander's first book Notes on th...Christopher Alexander's first book <i>Notes on the Synthesis of Form</i> was "discovered" by Ed Yourdon and Tom DeMarco and influenced their thinking on structured methods. You make a fair point about structured design methods as popularized by Yourdon, deMarco and others (including James Martin). But Alexander's approach had a lot of mathematical detail that the software gurus didn't try to replicate. While this book no longer reflects Alexander's own views on the design process, it is still worth reading.Richard Veryardhttps://www.blogger.com/profile/04499123397533975655noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-13132017288735227952009-10-05T06:22:22.177-07:002009-10-05T06:22:22.177-07:00@Roger: "I define complexity as the number of...@Roger: "I define complexity as the number of states in a system."<br /><br />Okay, we have a fundamental clash in terminology here, because the Cynefin or systems-theory concept of 'complexity' is radically different.<br /><br />In your model, the number of possible states is always knowable. It may be very large, but it is still possible for it to be predetermined in some way. (Presumably this is one of the key goals of the SIP methodology?)<br /><br />In Cynefin and the like, the key point is that the number of states not only is not known, but by definition <i>cannot be known</i> - i.e. is 'unknowable'. In many cases (as in some aspects of my radioactive-decay example) it could well be infinite or near-infinite.<br /><br />The mathematics for handling a finite series are significantly different from those for handling an infinite or near-infinite series. For a mathematical function involving only finite series, we can identify an explicit 'solution'; for any function involving an infinity, the best we can achieve is an approximation that points towards (implies) one or more context-specific answers, for which the optimal 'answer' is highly likely to change over time.<br /><br />Your version of 'complexity' is in effect a special-case in which artificial boundaries have been applied in order to constrain the possible number of identifiable states for each factor. This is a useful tactic within the artificial constraint of a software-based system-of-systems. However, it can easily become problematic as soon as we touch the real world at either of that scale, such as seen in signal-theory (in which true complexity applies well before we reach quantum-uncertainty levels), or in any form of human/machine interface.<br /><br />So whilst I don't disagree with your definition of 'complexity', I would urge you to perhaps be more careful and explicit about its limitations.<br /><br />Hope this helps.Tom Ghttps://www.blogger.com/profile/13526969814702899986noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-39038175006185010362009-10-05T05:58:12.891-07:002009-10-05T05:58:12.891-07:00David Wright: I am just finishing off a 20 page wh...<b>David Wright</b>: I am just finishing off a 20 page white paper that I hope will fill in the missing details. It should be ready within a few days. <br /><br />As far as “High Cohesion, Low Coupling”, I am not <i>proposing</i> high cohesion, low coupling, I am <i>assuming</i> it. One way to look at SIP is that it is driving an architecture which gives the ideal balance of high cohesion, low coupling. <br /><br />SIP does not address the solution architecture (how a subset of functionality is designed, architected, or implemented). It focuses on making sure we have the right number of subsets and the right functionality in each. In other words, SIP is about the optimal way of organizing a large system into smaller, autonomous systems. <br /><br />You could argue that the SIP focus is just a small part of the bigger problem of delivering an IT system. After all, Once you have these subsets identified, you have the main part of the task still in front of you, and SIP offers little here. <br /><br />I completely agree that SIP addresses only a small part of the puzzle of delivering an large IT system. But I believe it is a crucial part, the complexity management. Get this part right, and the rest flows smoothly. Get this part wrong, and the rest will likely fail. <br /><br />So SIP differs from a methodology like TOGAF, which addresses every aspect of architecting an IT system. SIP only tries to add value in one small but crucial area: complexity management.<br /><br /><b>Tom G, Jurgen Appelo, Richard Veryard</b> : The goals of SIP and structured design are similar, perhaps identical. However structured design has no methodology to ensure that the eventual solution is the least complex solution. It is driven by the skill of the architect. And, as I said in the original post, this is highly unlikely to drive the simplest possible solution. <br /><br />I assume that <i>complicatedness</i> has to do with the difficulty to understand a system. I define <i>complexity</i> as the number of states in a system. Complexity and complicatedness are thus related, but still quite different. Consider a bowl containing a single dice. Now add one more dice. By adding one more dice, you have increased the complicatedness of the system by two, but the complexity (the number of states) by six.<br /><br /><b>Itay Maman</b>: I think that we <i>can</i> measure system complexity apriori. What I claim for SIP is exactly this: to predict the overall complexity of a completed system before the first line of code is written and before the first architectural design is laid out for that system. Now obviously I can’t predict all aspects of the system complexity, in particular, I can’t predict how complex the implementation code will be (in terms of loops, branches, and organization.) But I claim that I can predict the inherent complexity of a proposed architecture for an system and thus the "goodness" of the proposed architecture before the architecture even begins.<br /><br />SIP is just the first step on the long journey of delivering an IT system. But it is a crucial step, and SIP makes sure you are heading in the right direction. <br /><br />Thanks, everybody for commenting. This is helping me hone my thoughts on this.<br /><br />As I said, I plan on a larger white paper that I hope will tie together the many pieces within a few days.Roger Sessionshttps://www.blogger.com/profile/16946430426943308823noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-27608423417846789772009-10-05T02:33:03.001-07:002009-10-05T02:33:03.001-07:00I agree that we need to be able to manage complexi...I agree that we need to be able to manage complexity and strive for solution that minimize it. Given that we cannot measure complexity a-priori, our best bet is to start building a solution and then gradually refine it. This is what Agile is all about (and presumably SIP, which I would like to hear more of). <br /><br />However, the architectural complexity (of software) is much more complicated that what was presented here. In software every composite compoenent (that is: soemthing that is made of smaller building blocks) is itself a building block. <br /><br />Thus, when building a solution for 10 business functions, one should also consider solutions where an individual function is taken apart and implemented by several different subsystems.<br /><br />For instance, if we decompose F1 into F1a and F1b we can then come up with the following solution: <br />S1: F1a, F2 - F8<br />S2: F1b, F9<br /><br />Clearly there are many other viable combinations which implies that the space of solutions is much bigger that what is implied in the post. This, of course, does not contradicts this post's main point, just strengthens it.Anonymoushttps://www.blogger.com/profile/15900841850889743147noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-87444190574505434672009-10-05T02:03:24.317-07:002009-10-05T02:03:24.317-07:00Sent TomG a couple of references via Twitter, but ...Sent TomG a couple of references via Twitter, but I thought I'd put them here as well. <br /><br /><a href="http://c2.com/cgi/wiki?NotesOnTheSynthesisOfForm" rel="nofollow">http://c2.com/cgi/wiki?NotesOnTheSynthesisOfForm</a><br /><br /><a href="http://sinet.ca/patterns/index.php?title=Notes_on_the_Synthesis_of_Form" rel="nofollow">http://sinet.ca/patterns/index.php?title=Notes_on_the_Synthesis_of_Form</a>Richard Veryardhttps://www.blogger.com/profile/04499123397533975655noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-72143838727442467242009-10-04T23:39:04.724-07:002009-10-04T23:39:04.724-07:00I agree with Tom G. This is about complicatedness....I agree with Tom G. This is about complicatedness. Not about complexity.Jurgen Appelohttps://www.blogger.com/profile/01159223378289101264noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-43885683697279518672009-10-04T23:26:43.413-07:002009-10-04T23:26:43.413-07:00The principles of structured design were influence...The principles of structured design were influenced by Christopher Alexander's first book Notes on the Synthesis of Form. How does SIP compare with Alexander's early thoughts on methodology?<br /><br />Alexander has largely repudiated this early work, and Tom's thinking is possibly closer to Alexander's later work.Richard Veryardhttps://www.blogger.com/profile/04499123397533975655noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-22107914726231442862009-10-04T22:31:17.867-07:002009-10-04T22:31:17.867-07:00I would echo David here: I can't see any clear...I would echo David here: I can't see any clear difference between this and Structured Design, other than that you have an (unexplained) metric for 'complicatedness' and some (unexplained) method for working with that metric.<br /><br />What this still <i>doesn't</i> do is address true complexity. "All other things being equal", you say: but in a true complex context, not only are 'other things' <i>not</i> equal, even when they are they don't <i>stay</i> equal - complexity is dynamic, not static.<br /><br />If, as I suspect, you use the term 'complexity' to indicate 'degree of complicatedness' in a constrained, stable, Aristotelian-logic system - typical of the <i>internals</i> of highly complicated IT-based systems-of-systems - then yes, your description makes sense.<br /><br />But if you think this method will give you control over true complexity, you'd be falling for the same trap that many fell into with chaos-mathematics. A metric of unpredictability does <i>not</i> make the context more predictable: it makes the unpredictability predictable, but the context remains unpredictable because the unpredictability is an <i>inherent</i> characteristic of the context. (Example: radioactive decay. We can usually define the half-life to a high precision, by statistical means, but it still tells us nothing about when a single <i>specific</i> atom will split - and that split may be the critical factor in the system. In other words, a chaotic 'market-of-one' factor driving a complex system.)<br /><br />The <i>internals</i> of IT systems can often usefully be described in terms of degree-of-complicatedness. Using reductionist tactics is often the most appropriate approach there. But the social/technical context within which that IT-system operates will invariably include some aspects of true complexity. Attempting to use reductionist tactics to 'solve' true complexity is a guaranteed cause of failure, especially in the longer term. So we need to be <i>very</i> clear about which kind of 'complexity' we're dealing with in each context, and adapt our tactics accordingly.Tom Ghttps://www.blogger.com/profile/13526969814702899986noreply@blogger.comtag:blogger.com,1999:blog-7826786244968404549.post-67935830475593689272009-10-04T19:57:37.276-07:002009-10-04T19:57:37.276-07:00Yes, do explain more. Right now it sounds like Str...Yes, do explain more. Right now it sounds like Structured Design: High Cohesion, Low Coupling... and how does one measure in SCUs?David Wrighthttps://www.blogger.com/profile/05103379078232846587noreply@blogger.com