Cost of IT Failure

What does IT failure cost us annually? A lot.

According to the World Technology and Services Alliance, countries spend, on average, 6.4% of the Gross Domestic Product (GDP) on Information Communications Technology, with 43% of this spent on hardware, software, and services. This means that, on average, 6.4 X .43 = 2.75 % of GDP is spent on hardware, software, and services. I will lump hardware, software, and services together under the banner of IT.

According to the 2009 U.S. Budget, 66% of all Federal IT dollars are invested in projects that are “at risk”. I assume this number is representative of the rest of the world.

A large number of these will eventually fail. I assume the failure rate of an “at risk” project is between 50% and 80%. For this analysis, I’ll take the average: 65%.

Every project failure incurs both direct costs (the cost of the IT investment itself) and indirect costs (the lost “opportunity” costs). I assume that the ratio of indirect to direct costs is between 5:1 and 10:1. For this analysis, I’ll take the average: 7.5:1.

To find the predicted cost of annual IT failure, we then multiply these numbers together: .0275 (fraction of GDP on IT) X .66 (fraction of IT at risk) X .65 (failure rate of at risk) X 7.5 (indirect costs) = .089. To predict the cost of IT failure on any country, multiply its GDP by .089.

Based on this, the following gives the annual cost of IT failure on various regions of the world in billions of USD:

REGION        GDP (B USD)  Cost of IT Failure (B USD)
World         69,800       6,180
USA           13,840       1,225
New Zealand   44           3.90
UK            2,260        200
Texas         1,250        110

Sessions's Complexity Aphorisms

- Complexity is like heat. We need just enough to solve the problem. Any more kills us.

- The best IT solution is the simplest IT solution that solves the business problem.

- The simplest IT solution is the NULL solution, but that fails the effectiveness test.

- Complexity is the enemy.

- The cloud is a platform. Not a complexity solution.

- 60% of all IT budgets are invested in projects that are at risk for unnecessary complexity.

- Occam's Razor Applied to IT: When you have two competing architectures that solve exactly the same business problem, the simpler one is the better.

- Consulting organizations perpetuate complexity through fear: This project is too risky. You will fail and be blamed. Let us fail and be blamed.

- IT Complexity is a tax paid by everybody that benefits nobody.

Three Rules for IT Simplification: Small, Separate, and Synergistic.

The most important factor in building a successful IT project is simplification. Simplification means that the overall system should be as simple as it can possible be while still delivering the goals of the project.

In general, there are three rules you must follow to build a simple IT system.
- Keep it small.
- Keep it separate.
- Keep it synergistic.

Let's take a look at each of these.

Rule 1. Keep it small.
Keeping it small means that a system should have no more functionality than it needs to deliver its defined goals. Every ounce of IT functionality must be clearly traceable to the business goals of the project. If a function is not traceable, it should be removed.

This, of course, can only be done if the business goals are clearly understood. Usually they are not. So make sure that at the beginning of the project, the business folks and the IT folks have clearly documented each business goal of the system. And that every bit of planned functionality can be mapped back to one of those goals.

This goal is often at odds with people's desire to make systems "reusable". In order to make systems more "reusable", IT likes to make systems more configurable and more functional. IT often convinces the business that reusability is an important goal, thereby justifying their efforts. Efforts to achieve reusability almost always add unnecessary complexity, and, in the end, rarely succeed. Think long and hard before you sacrifice simplicity at the alter of reusability.

Rule 2. Keep it separate.
An IT organization will inevitably need many systems and these systems will need to interoperate. Interoperability is at odds with simplicity. From a simplicity perspective, the less interoperability between systems, the better. From a corporate perspective, the more interoperability between systems, the better.

You can see the conflict. It is important to strike a balance between these two objectives. Design as much interoperability as necessary, but no more than necessary and design it with as much respect as possible for the boundaries between systems.

Rules 3. Keep it synergistic.
One of the most critical issues, from a simplicity perspective, is how functionality is placed. Many systems suffer huge complexity problems because functionality that should be co-located is not, or, just as serious, functionality that should not be co-located, is. It is critical that functions be placed based on natural (that is, business-defined) synergies.

So if simplicity is your game, remember the three S's that govern simplicity: small, separate, and synergistic. You won't be sorry!

How Many Enterprise Architects Does It Take To Measure A Donkey?

I don't know how this got started, but we were tweeting a discussion about enterprise architecture, and somehow the question came up...

How Many Enterprise Architects Does It Take To Measure A Donkey?

A: depends on where the datum is and which part of the donkey they measure!

A: and should not speed of donkey be considered too? For relativity effects I mean.

A: ahh yes, but it is velocity and in the direction of measurement! A jumping donkey!

A: jumping donkeys - can I have some of what you two are on?

A: EAs measuring the donkey? One measures required height of TO-BE donkey, one argues length and height are basically the same, ...

A: .. one says the problem is that the front half of the donkey and the rear half are not in alignment.

A: ... one says that we can't measure the donkey until we have completed the business case.

A: one say that we need an industry consortium to define the best practices for donkey measuring.

A: ... one says we don't need to measure the donkey. It's not strategic. We need to outsource it.

A: ... one says that we can't measure the donkey until we first partition it into little pieces so that it isn't so complex.

With contributions from @RSessions, @richardveryard,@seabird20, @taotwit, and @j4ngis. (Did I miss anybody?)

Factors Driving System Complexity

IT Systems are failing at an alarming rate and the rate of failure is increasing. Approaches that we have used in the past to successfully deliver IT systems are no longer working.

Something has changed: the complexity of the systems. It should be no surprise to anybody in IT that the complexity of the systems we are being asked to build has dramatically increased. The only surprise is that we have not adapted the processes that we use to build IT systems to take into account this increase.

Processes that can drive simple IT systems development do not scale up as the complexity of the systems increases beyond a certain point, a point that we have long passed.

I do not advocate throwing out what we know about building simple systems. Approaches such as Agile Development are great, as long as the complexity of what we are building is manageable. The approach that I advocate is learning how to break large, complex systems that we do not know how to build into smaller, simpler systems that we do know how to build.

I call this process partitioning. It is important to remember that the goal is not just to create smaller systems, but smaller systems that are as simple as possible. The specific process that I advocate for doing this is called SIP, for Simple Iterative Partitions. SIP is designed from the start to generate systems that are both small and simple.

While the goal of partitioning is to create smaller simple systems, the process itself is neither small nor simple. Planning for simplicity paradoxically adds complexity to the planning process but pays huge dividends further on. The extra work involved in delivering complex systems is far greater than the extra work involved in planning for simplicity from the beginning.

Partitioning is a science, like medicine. It must be led by people who understand the nature of the disease and have been trained in how to manage it. The training is important because there are many things that can go wrong. When things do go wrong, complexity is inadequately removed, or, in some cases, made worse.

IT failures due to poor partitioning are epidemic in our industry. For example, there are many failed service-oriented architectures (SOAs). Behind almost all of these SOA failures is a partitioning failure.

I have been studying IT failures for a number of years now. I have concluded that partitioning failures are the primary cause of most IT failures. The bigger the failure, the more likely it is that incorrect partitioning is the root cause. The symptom of partitioning failure is always the same: the project seems swamped by complexity. If you have worked on a project whose complexity killed the project, then you have experienced firsthand the effects of partitioning failure.

I have seen so many partitioning failures that I have started to recognize underlying patterns or categories of partitioning failures. At this point, I have identified eleven categories, and the list is growing as I analyze more failures. The eleven categories are as follows:

• Delayed partitioning: The partitioning did not begin early enough in the project life cycle. It should be mostly complete before the business architecture is begun.
• Incomplete decomposition: The decompositional analysis of functions did not go far enough, resulting in too coarse a granularity of the functions being partitioned.
• Excessive decomposition: The decompositional analysis of functions went too far, resulting in too fine a granularity of the functions being partitioned.
• Bloated subsystems: Too many functions have been assigned to one or more subsystems.
• Scant subsystems. Too few functions have been assigned to one or more subsystems.
• Incorrect assignment: Multiple functions have been assigned to the same subsystem that do not belong together or have been assigned to different subsystems that do belong together.
• Duplicated capabilities: Functions have been duplicated across different subsystems.
• Unnecessary capabilities: Functions have been partitioned that should not have been included in the first place.
• Technical partition mismatch: The technical partitioning does not match the business partitioning, a common problem in purchased systems and service-oriented architectures.
• Inadequate depth: The technical partitioning does not extend far enough. In SOAs, this problem often manifests itself as multiple services sharing common data.
• Boundary decay: The boundaries between subsystems in the partition are not strong enough, and functionality slips back and forth between subsystems.

As you can see, there are many opportunities for errors in complexity management. And complexity (and therefore IT failure) is highly sensitive to partitioning errors, much more so than to implementation errors. It is critical, therefore, to have a well defined process to guide you and check your results along the way.

(Thanks to Chris Bird for suggesting this topic with his innocent tweeted question: “What are contributors to complexity?”)

Adaptability of Large Systems

As I was tweeting the other day, I came upon a tweet from Noel Dickover about large systems being less adaptable than small systems. This began a tweet exchange that I thought brought up some important issues about adaptability.

Many people believe that large systems are harder to adapt than smaller systems. And, in genera, this is true. But system adaptability is not a function of size, it is a function of architecture. When a large system is partitioned correctly so that it is composed of a number of smaller, autonomous systems, then it MAY be more adaptable than a single smaller system.

I say "may", because its adaptability depends on how well it has been partitioned. The key is not whether the partitioning is good technically (say, mapping well to an SOA), but how well the technical partitions overlay on top of the business partitions of the organizations.

In other words, when a large system is built of autonomous smaller systems AND those smaller systems map well to the autonomous processes that occur naturally within the business, then, and only then you have a large system which is highly adaptable.

The reason so many systems fail to achieve this (even when they do manage a reasonable technical partitioning) is that the technical partitioning MUST BE driven by the business partitioning. This requires a partitioning analysis of the business that is completed before the technical architecture of the system is even begun.

This business partitioning analysis is best done by representatives of both the business and the IT organization. The business group has the best understanding of how functions relate to each other and the technical group has the best understanding of how this business partitioning analysis will eventually drive the technical partitioning architecture.

Since both business and technical experts are involved in this exercise, I place this work in the common ground between business and technology, the watering hole that we call enterprise architecture. But it is enterprise architecture with a very specific focus: driving technical partitioning from business partitioning analysis with the eventual goal of highly flexible systems that are pegged closely to the business need and mirror closely the business organization.

The Cancer of Complexity

After many years in IT, I am convinced that code complexity is a relatively unimportant issue. This may sound strange, coming from someone who is always reminding people that Complexity is the Enemy. How can I not care about code complexity?

What is much more important than code complexity is how that code is architecturally organized. We can deal with complex code if that code is well sequestered. It is the effectiveness of the sequestering rather than any pockets of code complexity that will determine the complexity of the larger system.

Complex code that is well sequestered has what I describe as "benign complexity". An example of a system with benign complexity is a web service that is poorly written (complex) but well encapsulated (sequestered). IT systems that have benign complexity may have localized problems, but these problems are rarely lethal to the system as a whole, and, if they can't be solved, can at least be surgically excised.

Complex code that is poorly sequestered has what I describe as “malignant complexity”. An example of a system with malignant complexity is collection of services that all share common persistent data. These systems are also complex, but now the complexity is not localized and is almost impossible to address. These systems are usually headed for serious problems.

There are many similarities between malignant cancer (that is, cancer that is progressive and uncontrolled) and malignant complexity. Here are some that I have identified in recent twitter conversations:

• Both malignancies grow exponentially over time.
• It is difficult or impossible to control the rate of growth. (Thanks to A. Jangbrand for this one.)
• Both malignancies can be prevented much more easily than cured.
• Left to their own devices, both malignancies will destroy their host.
• When removal is attempted, it is easy to splinter the malignancy and form new malignancies that can themselves grow. (Thanks to Richard Veryard for this one.)
• By the time symptoms are noticed, the malignancy has often reached an advanced and sometimes incurable state.
• Both malignancies can spread to other locations that are only remotely connected with the original location.

So how complexity is organized, benign or malignant, is more important than the degree of complexity itself. Standard code complexity analysis approaches, such as cyclomatic analysis, fail to take into account how code is organized. This is why these metrics are rarely useful in any kind of large scale complexity analysis. They may tell us that a function, perhaps even an application, is complex, but they never tell us whether that complexity is limited to the function or application, or is about to spread to the larger system.

I do believe that complexity is the enemy. Until we better understand complexity, our chances of building better IT systems is limited. The first thing we must understand about complexity is that not all complexity is equal. And the complexity on which most people focus is probably the least complex complexity of all.

(Join the twitter conversation - @RSessions.)