The Cancer of Complexity

After many years in IT, I am convinced that code complexity is a relatively unimportant issue. This may sound strange, coming from someone who is always reminding people that Complexity is the Enemy. How can I not care about code complexity?

What is much more important than code complexity is how that code is architecturally organized. We can deal with complex code if that code is well sequestered. It is the effectiveness of the sequestering rather than any pockets of code complexity that will determine the complexity of the larger system.

Complex code that is well sequestered has what I describe as "benign complexity". An example of a system with benign complexity is a web service that is poorly written (complex) but well encapsulated (sequestered). IT systems that have benign complexity may have localized problems, but these problems are rarely lethal to the system as a whole, and, if they can't be solved, can at least be surgically excised.

Complex code that is poorly sequestered has what I describe as “malignant complexity”. An example of a system with malignant complexity is collection of services that all share common persistent data. These systems are also complex, but now the complexity is not localized and is almost impossible to address. These systems are usually headed for serious problems.

There are many similarities between malignant cancer (that is, cancer that is progressive and uncontrolled) and malignant complexity. Here are some that I have identified in recent twitter conversations:

• Both malignancies grow exponentially over time.
• It is difficult or impossible to control the rate of growth. (Thanks to A. Jangbrand for this one.)
• Both malignancies can be prevented much more easily than cured.
• Left to their own devices, both malignancies will destroy their host.
• When removal is attempted, it is easy to splinter the malignancy and form new malignancies that can themselves grow. (Thanks to Richard Veryard for this one.)
• By the time symptoms are noticed, the malignancy has often reached an advanced and sometimes incurable state.
• Both malignancies can spread to other locations that are only remotely connected with the original location.

So how complexity is organized, benign or malignant, is more important than the degree of complexity itself. Standard code complexity analysis approaches, such as cyclomatic analysis, fail to take into account how code is organized. This is why these metrics are rarely useful in any kind of large scale complexity analysis. They may tell us that a function, perhaps even an application, is complex, but they never tell us whether that complexity is limited to the function or application, or is about to spread to the larger system.

I do believe that complexity is the enemy. Until we better understand complexity, our chances of building better IT systems is limited. The first thing we must understand about complexity is that not all complexity is equal. And the complexity on which most people focus is probably the least complex complexity of all.

(Join the twitter conversation - @RSessions.)