My specialty is understanding how to measure, locate, and eliminate unnecessary IT complexity. One of the difficulties of this field is terminology. You would think that a word like simple would be simple to define. But there is no widespread agreement on what the words related to complexity mean.
So in this blog, I am going to define some of the terminology that is important in IT complexity reduction. I won't claim that these meanings are relevant to all fields, but when I use these words, this is what they will mean. This is my initial pass. I will give examples from my own work, since that is the domain with which I am most familiar.
I invite readers to comment. Soon I will consolidate the discussion into a terminology proposal for www.cuec.info (The Consortium for Untangling Enterprise Complexity.) I assume that better definitions will come out of this discussion.
Simplicity Framework: A simplicity framework is an approach, model, or methodology that seeks to find, measure, and remove complexity from some domain. Any simplicity framework should specify the domain for which it is intended. Example: SIP (Simple Iterative Partitions) is a simplicity framework for IT architecture.
Complexity: Complexity is a measure of an attribute of a system that makes that system difficult to understand, manage, and/or implement. Complexity is measured in some units appropriate for the domain and defined by a simplicity framework. Example: In SIP, complexity is measured in Standard Complexity Units (SCUs).
Solution Set: For some problem P in a given domain, the solution set is the set of all solutions that solve P. Note that a solution set only includes solutions to P. If a proposed solution would not solve P, then it is by definition not a member of the solution set. Example: When considering architectures for an inter-library loan system, we examined dozens of potential candidates from the solution set.
Simple Solution: A simple solution is the element of a solution set that has the least complexity of all elements in the solution set. Note that it is theoretically possible that more than one solution from the solution set share the lowest complexity. In this case, there are multiple simple solutions. Example: People were surprised that the simple solution to the inter-library loan system was not the one initially proposed.
Complex Solution: A complex solution is any of the elements of the solution set that are not the simple solution and whose complexity can be reduced through the simplicity framework. Example: In the inter-library loan system, all of the originally proposed SOA architectures were complex solutions.
Simplistic Solution: A simplistic solution is any proposed solution to a problem that lacks the minimum complexity necessary to solve that problem. By definition a simplistic solution is not a member of the solution set. Example: The catalog system we looked at turned out to be a simplistic solution to the inter-library loan system.
Chaotic Solution: A chaotic solution is a solution that is presumed to solve a problem, but whose complexity is so high that using (or continuing to use it) is not feasible and reducing it is not practical given the simplicity framework. Note that it is not always possible to determine if a chaotic solution is even a member of the solution set. Example: The present inter-library loan system is a chaotic solution.
Do you have any ideas for more terms that should be defined? Do you have issues with these definitions? Leave comments here or discuss with me on Twitter (@RSessions). And keep an eye on www.cuec.info for efforts to standardize some of these terms.
Planned Version 2 Changes:
@johanlindberg suggested using candidate to describe Simplistic and Chaotic. His argument: Simplistic and Chaotic are by definition not solutions. Good point! He also suggested giving comparison terms to show similar terms in complexity theory and SW development.
@HotFusionMan (Al Chou) suggested adding note about layers of complexity. I think this is a detail of the simplification framework, but I need to make clear the generic character of these definitions.
Add definitions for simplicity, simplification. Add description of which problem spaces these definitions are appropriate for (those that are striving for simplicity.) Make less specific to IT and general to any situation in which a move from complex to simple is desirable.
Incorporate Nikos's discussion on understandability.
And thanks to @JohnDCook for Twitter contributions!
Subscribe to:
Post Comments (Atom)
9 comments:
Any solvable problem which is in a chaotic situation can be solved by a complete side-by-side replacement. The cost of preparing an transparent changeover may be much more than the cost of the solution however. The thing that makes some chaotic situations unresolvable is when the cost of solving it is more than the cost of leaving it unsolved.
Complexity science defines a complex system as self-organization at the edge of chaos and involves any system with multiple interacting variables so that linear calculations don't work. There is a large literature in this field. The effects of variable changes in a complex system can become nonlinear, trending the system towards potential chaos, or exponential success. To understand this kind of system people use the modeling methods of agent-based, system dynamic or machine learning models. There is nothing wrong with seeking simplicity except that it is not really available in any large system. What is available is a simpler representation or interface to the system. Complex is not the same as complicated. A complex system can be highly robust and also understandable/modelable; a complicated system is inevitably fragile and is hard to understand. When management takes a complex system and tries to force it to be simple, the likely outcome is chaos, because the oversimplified system likely introduces turbulence. If you don't understand the amount or interactivity of the variables you can indirectly create conflict among them. BTW the latter is the problem that SCRUM and Agile are meant to solve: contain change to time limited team-organized segments of work so that course changes from above continually interrupt a process and lead to incomplete, unconnected and possibly conflicting solutions.
I meant "so that course changes from above don't continually interrupt..."
I don't think Roger is using the term "complex" in the same way that complexity theory (was called chaos theory) uses it. In enterprise architecture, self-organisation only occurs as a result of a failure of the business to manage or coordinate itself centrally. Problems of that kind are intrinsically political problems, and I don't think that's Roger's focus.
cjh: When I described "chaotic" I was describing the solution, not the problem. So we are in agreement that the way to deal with a chaotic solution is to find a simple solution to the same problem.
Oleander: Complexity science is mainly interested in understanding how complex systems behave as them become more complex. I know how IT systems behave as they become more complex: they fail!
I am interested in using an understanding of how systems become complex to prevent them from becoming complex.
There are a few people in complexity science studying how to reduce complexity (AKA make things simpler) but they are few and I believe I am one of the few trying to do this in a systematic and reproducible way. If you know any others, let me know!
Problems in simplification only occur when we fall past "simple" and into the simplistic domain. The good news is that this is relatively rare. Most problem come from marching in the other direction: from simple to complex to chaotic.
And I don't expect my terminology to match those looking at complexity from a completely different perspective. That is why I took the trouble to define the terms as they make sense in my problem space.
As far as SCRUM/Agile, they are useless in the chaotic domain and almost useless in the complex domain. It isn't until we land in the simple or near simple domain that these methodologies offer real value.
Clifford Heath: You are right in that self-organization is EA or IT is a bad thing. I am looking for directed organization, not self organization. Thanks!
Roger, most definitions above can be traced back to definition of complexity. Simplicity is removal of complexity. In some sense, simplicity is adequate lack of complexity. Instead if the definition of simplicity can be more direct in terms of attributes of simplicity, in my opinion, would help people find simple solutions more easily and eventually more naturally.
Simon-Ando have shown that 4 essentially things needed for stability of large-scale systems - a) hierarchical abstractions b) strong interactions within subsystems but week inter subsystem interactions c)stability of subsystem weakly dependent on stability of other subsystems and d)global equilibrium(stability) is dependent on equilibrium of all subsystems
I have always thought though the human mind works well with abstractions, when we create IT systems, we are so bogged down by details that we lose the abstractions. This is perhaps the first step in creating complex solutions.
These definitions are great and helpful. I was thinking "how would I 'action' these definitions to create simple solutions?"
Hi Roger, after reading your post I wondered what if any variability in the domain (the problem) should be considered? Can a complex solution in one domain become less complex when that domain changes? For example, in high-frequency trading of financial instruments, the last 5 years have resulted in non-functional requirements for latency moving from second to sub-second to sub-millisecond and for throughput moving from tens of thousands of messages per second to millons of messages per second. Would a hardware based architecture been classified as complex 5 years ago been considered more simple once the business problem began to change as fast as it did?
IMHO Methodologies, models approaches and frameworks are all different things. Interestingly I’ve just checked the definition of “framework” and I wasn’t particularly happy with many of them (I must be hard to please!). The first entry for framework defined “Model” as the “hypothetical description of a complex entity or process”.
Do you see what you’re doing (or what you’re aiming for) as a combination of useful models and processes – with a unified taxonomy that covers the whole concept?
RE "Simple Solution": If it’s the “best” solution (it has the least complexity) is it not the “simplest”? The same also applies for “Complex Solution”.
Also, what defines whether or not a solution is simple or complex? Is it an arbitrary measure, an absolute one or a relative one? It’s possible that the simplest option in a solution set may still be too complex.
Do you need a definition for solutions which are in the middle ground (assuming there is such a thing in your model); is there any ambiguity you need to potentially identify?
RE "Simplistic Solution": Given that the focus of your framework / model is complexity (and therefore simplicity) I think you might want to avoid using the words simple and complex (and their derivatives) in any other context, so as to avoid terminology collision and to get a cleaner taxonomy.
Looking more closely at your definition – is it that the solution is too “simple” (as in dumb) or that it simply doesn’t meet the requirements? I’d say the latter; assuming you didn’t specifically want to address “dumb” solution options. Would a solution ever be too complex?
The way I read this is that a solution isn’t actually a solution because it simply can’t do what’s expected.
I see how you might want a “simplistic solution” to balance the chaotic one, but I wonder if the term simple is too loaded in the context of your framework.
I agree that we need to consider - while creating IT systems -the abstract that the human mind relates to and feels comfortable with - cause the human experience from our developed interface is what counts in the end.
Post a Comment