When it comes to IT failure, there is no lack of "the usual suspects". We can look to failures in project definition, project management, and needs assessment. We can point to flawed architectures, implementations, and testing procedures. We can focus on communications failures between the business and IT, between IT and the user community, and between different business units.
Yet given this extensive collection of failure factors any of which can doom an IT project, why do I focus almost exclusively on the issue of complexity?
I see all of the failure factors as falling into one or more of three categories:
1. The factor is caused by complexity.
2. The factor is greatly exacerbated by complexity.
3. The factor is solved as a side effect of solving the complexity problem.
Examples of failure factors that are directly caused by complexity are the various technical failure factors, such as poor security or scalability. It is very difficult to make a complex system secure or scalable. Solve the problem of complexity, and these problems become much easier to solve.
Examples of failure factors that are greatly exacerbated by complexity include those related to communications. As a project increases in complexity, people tend to fall more and more into specialized jargon which makes communications more difficult and adds yet more complexity to the already complex project. Different groups tend to see each other as the enemy. As a side effect of learning to solve complexity, the groups learn to see not each other as the enemy, but complexity as their common enemy. Their common language becomes the language of simplification.
Examples of factors that are solved as a side effect of solving the complexity problem include those related to organization. For example, a well known failure factor is the lack of executive sponsorship. But it is difficult to find sponsors for large complex expensive projects. Once a project is broken down into small, simpler, less expensive projects, finding executive sponsors for those projects is much easier.
The other reason I focus so much on complexity is that of all these failure factors, complexity is the only one that is universally present in all failed projects. The fact is that we are quite good at doing simple projects. Our skills just don't scale up to complex projects. So we can either tackle the failure factors piecemeal and try to figure out how to scale each up to higher levels of complexity, or we can try to figure out how to scale down the complex projects into simple projects that we already know how to solve.
So while I pay close attention to all of these failure factors, I continue to believe that the one that deserves our undivided attention is the problem of complexity.