Tuesday, March 13, 2012

The Equation every Enterprise Architect Should Memorize

In my white paper, TheMathematics of IT Simplification, I gave the following equation for calculating the complexity of an IT system

C = F3.11

where C is the complexity in Standard Complexity Units (SCUs) and F is the number of business functions implemented within the system. As I point out in the paper, F can be either business functions or dependencies on other systems. An SCU, by the way, is the amount of complexity in a single business function unconnected and unrelated to any other business function.

This equation is a simplification of a more complex equation that I called Bird's equation after a friend, Chris Bird, who first derived that more complex equation. This simplification was suggested by a woman reader who's name I have regrettably misplaced. If this person was you, please let me know so I can give you credit.

Although I was correct in my discussion about this equation in the white paper, I was a little vague on the proof. A friend, Bill Indest, challenged me on the proof. So with the help of another friend, Nikos Salingaros, I have decided to fully derive the equation, and go one better. Not only will I derive this equation, I will derive the more general, more useful form.

This equation starts with what I call Glass's Law which, as I explain in the white paper, is named for Robert Glass who popularized the law that increasing the functionality of a system by 25% doubles the complexity of the system.

So we know that complexity and functionality increase together, and complexity increases faster than functionality. This describes a proportional relationship between F (functionality) and C (complexity). The simplest such dependence is an exponential one: in other words, there exists some power X such that C = FX. Okay. Now let's solve for X .

We know from Glass's Law that when functionality increases by 0.25, complexity doubles. In other words,

2C = (1.25)FX

So now we have two equations, both of which must be true.

Eq. 1:  C = FX
Eq. 2:  2C = (1.25F)X

Now we need to move into logarithms. Recall that logW Y = Z is interpreted as Y = WZ. And also recall that if W (the base) is not specified, then it is assumed to be 10. So, for example,

log10 100 = 2, or we could just say

log 100 = 2, since 10 is assumed.

One of the laws of logarithms states that if

A = B, then
log A = log B

So let’s go back to our two equations:

Eq 1: C=FX
Eq 2: 2C = (1.25F)X

We can use this property of logarithms to transform equations 1 and 2 into their logarithmic equivalents:

Eq 3: log C= log (FX)
Eq 4: log 2C = log (1.25F)X

Now we use another two log identities:

log AB = B log A
log AB = log A + log B

Using these identifies, we can transform Equation 3 and 4 into

Eq 5:  log C = X log F
Eq 6: log 2 + log C = X (log 1.25 + log F)

We now subtract Eq 5 from Eq 6 giving

Eq 7: log 2 = X(log 1.25 + log F) – X log F

which is the same as

Eq 8: log 2 = X log 1.25 + X log F – X log F 

which reduces to

Eq 9: log 2 = X log 1.25

We can now divide both sides of Eq 9 by log 1.25, giving

Eq 10: log 2/log 1.25 = X

Since log 2 is a constant=3.01, and log 1.25 is a constant=0.097, then the first half of Eq 10 simplifies to 3.01/0.097, which is 3.11.

Thus X in Eq 1 is equal to 3.11, and complexity is related to functionality by

Eq 11: C = F3.11.

This, of course, was my original assertion. Now this equation assumes that you buy into the fact that complexity doubles when functionality increases by 25%. But if you don’t agree with this, you can replace the X by a value that represents your belief (or, even better, observation.)

Eq. 12: C = F (logC/logF)

Let’s say you observe that in your organization that complexity doubles when F goes up by 50%. Then your formula for complexity is

Eq. 13: C = F (log2/log1.5)

Which would be

Eq. 14: C = F (0.301/0.176) = F 1.71

Equation 12 is so important to understanding complexity in an enterprise that I will refer to it as the Fundamental Equation for Enterprise Complexity. Everything you need to know about enterprise complexity can be derived from this equation. The formula should be memorized by every enterprise architect.

See? Aren’t you glad you stayed awake?


Stephen Law said...

I am probably a bit off at a tangent, but if I reflect on a few things you say and reminisce back to a few multi-billion dollar projects I have worked on, a lot of time was spent breaking the problem down in to smaller pieces to help manage the project, e.g. chunking the system down into a number of sub-systems and defining a collection of services and not to mention, producing large amounts of PowerPoint.

As an Enterprise Architect my mind-set was often in classification and association mode in an attempt to understand the very complex problem space and give some structure to it. We would produce lots of PowerPoint with reference architectures and transition plans, yet we still did not know if we were dealing with a bird or a plane or what the end game really looked like and neither did the business.

I guess ultimately the aim of these very large projects and programmes was to build something, e.g. a new ‘castle’ or enterprise ultimately underpinned by a new information system, but often several years in and many hundreds of millions of dollars later, nobody really knew what it looked like.

So, whilst structure and partitioning is very important, having some visualisation of what the final system might look like can be useful and very simple tools like http://www.balsamiq.com/ can help (as well).

Stephen Law

Roger Sessions said...

I think partitioning the functionality into sub-systems is the critical point of the project, the point at which projects succeed or fail.

The problem is that there are an infinite number of possibilities for how to do the partitioning. Given the very large number of sub-optimal partitions and the very small number of optimal partitions, the chances for finding an optimal partitioning solution are highly stacked against you.

This, of course, is why I advocate using SIP, since it guides you directly to the optimal partitioning. It is the light leading you out of the forest.

Most people do EA (whatever that means for them) and then partition. I believe this is the wrong order. We should partition and then do EA. It is simply too difficult to do EA for a large, unpartitioned system.

SIP allows you to do the partitioning with only minimal knowledge about the system being partitioned. I cover this in the White Paper, The Mathematics of IT Simplification. The result of this is that you can partition first and then do EA, the only order I believe can be successful.

Stephen, if you are interested in this and would like to discuss offline, just let me know.

- Roger Sessions

IT project quality said...

Your blog is quite funny. Are you expecting us to really look at your equations carefully or are you just joking about these? Surely IT projects can be complex, but if you are serious, I think perhaps you are trying to quantify things too much? And what would be the objective of quantifying IT system complexity? To do what? ensure enough resources? Ensure smart people working on the project? I guess I can go through your equations, but might take a while!

Anonymous said...

I have been researching Enterprise Architect for 1 year, I applied to an internal ETA position with my current employer (last week). Every time I do my research I end up back to one of your blogs, websites or white papers.

We do not have a current EA program so if I am selected for the position (currently a solutions Architect - previous systems Architect) I will be task with selecting frameworks, identifying tools and ultimately building the EA.

I previously read about your SIP and I am fascinated by it. Am I developing an EA or an EA framework? I guess with all the variants I will end up using a hybrid of(TOGAF 9.1)framework to build our EA.

With the Business domain being arguably the most important and this being my weakest area of expertise I need all the tools and assistance I can get to ensure my success.

So with a company size in the billions can I / should I start on the business domain and begin SIP with it or am I missing something?

I am on my math skills so the simple equation was a little over the top - lol


Roger Sessions said...

Hi Brian,
SIP is really about reducing the organizational complexity that typically chokes large EA projects. In one sense, it isn't an EA framework at all, but a framework for managing EA complexity. Most EA projects take so long to finish that they are obsolete before they are completed. This is the problem SIP addresses.

I'd be glad to discuss this with you. Let me know if you would like to have a chat about this.

Best wishes,

Matt said...

can you fix the typo in the equation 2C = (1.25)F^X --- the close bracket is placed incorrectly


Matt said...

Eq 12 is not quite accurate. I think you need to introduce new variables...

iC = multiplicative increase in C
= 2 for Glass's Law

iF = multiplicative increase in F
= 1.25 for Glass's Law

Then Eq 12 is better expressed as
C = F ^ (log iC / log iF)

Roger Sessions said...

Matt, good suggestions, thanks! Can you add your full name, so I can credit you with these corrections?

Matt said...

"Matt" is Matthew C. Clarke

Anonymous said...

Complexity while interesting can not be used in isollation to tell us much about anything, a badly chosen, or written simple system can be as damaging to an organisation as a badly chosen or written complex one.

I have to make issue with the notion of not considering the enterprise first, dealing with lower order parts partitions must introduce the risk of over working iun needed elements or designing a camel ( with two humps to take advantage of re use) when a horse was needed.

enterprise architecture gets too much codification and taxonomy and not enough focus on the strategic goal of organising business for success.

I have been creating business and IT strategy for nearly twenty years, used many bits of many approaches and architectures and there isn't a single one that suits all cases and more than that the influence upon the out come is a human one not a mathematical one.

In respect of the placemennt of braces in the equation A*X^n is the same as (AX)^n and A(X^n) no braces are needed in a one term factor

Unknown said...

@Matt: Eq 4's parentheses usage is conceptually correct. Mr Sessions is introducing the two equations as follows:

Eq 1: exponential relation for initial condition (step 0) -> C = F^X

Eq 2: exponential relation for next iterative step, as expressed by Glass -> 2C = (1.25*F)^X

Also, in a subsequent reply, you state A*X^n = (AX)^n. I think you need to review your polynomials, because this is categorically false.

Lastly, I think what Mr Sessions is trying to do is very similar to what I've been taught in SW engineering theory: Formal Specification. The concept to analyze a program rigorously with maths and logic.

So you can espouse your many years of experience using qualitative concepts (and I'm sure they've served you well) but I find this application of rigorous analysis of systems complexity to be quite refreshing.