In reading Michael Krigsman's blog, I was introduced to Senate bill S.3384, "Information Technology Investment Oversight Enhancement and Waste Prevention Act of 2008." The bill was placed on the Senate Legislative Calendar on 10/1/08.
This bill was introduced by Senator Thomas Carper (D-DE) and cosponsored by Sen. Norm Coleman (R-MN), Sen Joseph Lieberman (ID-CT), Sen. Susan Collins (R-ME), Sen Claire McCaskill (D-MO), and Sen. George Voinovich (R-OH). The bipartisan support (two democrats, two republicans, and an independent) in itself makes the bill noteworthy.
The bill requires that any government IT investment project that has "significantly deviated" from its projections must be reported up the command chain, ultimately ending with the responsible CIO. Any project that is off its projections by more than 20% is considered "significantly deviated." Projects that are in "gross deviation" are further reported to the appropriate congressional committee. "Gross deviation" is considered more than a 40% deviation from its projections.
Deviations are measured relative to the Earned Value Management (EVM) benchmark. This benchmark includes timely measures for project expenditure, project completion, and project deliverables.
S.3884 requires that this reporting be done on a quarterly basis. This means that deviations must be measured (and reported) long before the project is actually completed.
State governments often follow the lead of the federal government, especially in "watch-dog" type legislation. Therefore it seems likely that S.3884 may have an impact beyond just the federal government and into all areas of the public sector.
EVM benchmarks are highly dependent on the ability of an organization to accurately forecast both deliverables and costs. These are both areas in which the government has historically been weak. I have frequently written about governmental IT projects that have gone under-promise, over-budget, past-due, or all of the above. If S.3884 is going to be successful, it will require a major new approach in how government does IT.
The biggest problem that the government has with IT is its chronic failure to manage complexity. The more complex a project, the harder it is to project cost and predict deliverables, two absolute requirements for an EVM analysis. There is no possible way that S.3384 can be implemented successfully unless the government first takes steps to understand and manage IT complexity.
I strongly support any requirement that the government do a better job with IT. But I think the Senate needs to be realistic. Passing S.3384 without giving governmental IT bodies the tools they need to address IT complexity is setting them up for failure.
Let's phase in S.3384. First, introduce SIP (or some other equivalent methodology) to the government to show how highly complex systems can be organized as much smaller, simpler, autonomous systems. Then hold the government accountable for how it delivers those simpler systems.
If the government takes the two steps of IT complexity management and IT accountability. we can transform how the government does IT. We will have a government that delivers IT projects on-time, on-budget, and meeting the needs of the citizens that pay for those projects. That is you and me.
Tuesday, November 25, 2008
Wednesday, November 12, 2008
Carving up an Enterprise Architecture
Melissa A. Cook recently published an article in ComputerWorld titled Enterprise Architecture: Lessons from the cutting board. In this article, Melissa likens developing an enterprise architecture to cutting up a chicken for dinner. She gives three lessons for creating an enterprise architecture.
Lesson 1: Cut through the joints, in other words, look for natural boundaries around business processes. As she points out, “if you get the natural boundaries right, it simplifies the number and complexity of interfaces. How do we do this? Melissa says “we need to take a gander at the enterprise goose, the whole thing.” A good check point is looking at data: “a ton of data elements or columns of data on an interface, you probably have the boundaries wrong.”
Lesson 2: Natural breakpoints or boundaries are similar for similar organizations. In other words, see what others that are in similar businesses have done and use this for a model.
Lesson 3: Boundaries don’t change based on size of the organization. Big turkeys carve much like little chickens.
I agree with Melissa strongly on two points. First, if you get your enterprise architecture right, you will make a great leap forward in managing the complexity of your systems. Second, your success in getting the architecture right is reflected in the data structures moving around your system.
Having said that, I feel several points need more comments. Let’s start with her thoughts on data. While I agree that poor enterprise architectures result in complex data patterns, I disagree that this is a good test to use to evaluate your enterprise architecture. The reason is simple. By the time you can notice the problems with data, it is far too late to do anything about the problems with the architecture. You have already implemented your systems and any changes in the enterprise architecture at this point are going to be very expensive.
So what is the solution? Any enterprise architecture should be tested against theoretical models for enterprise architectures based on the mathematical laws of complexity. These tests can be conducted BEFORE the architecture is implemented and can pinpoint problem areas before money is spent on their implementation at a time when they are still relatively inexpensive to fix.
Another area that we need to look at more closely is her lesson 2. While it is true that similar organizations have similar boundaries, this is not particularly helpful information. Why? Because “similar” is not good enough. Even small errors in “cutting up the chicken” can lead to huge problems with complexity. I personally have seen millions of dollars lost due to a single poor function placement. Again, the answer is to methodically check each function for correct placement, not based on a particular chef’s preferences, but based on the strict mathematics of synergies.
Don’t get me wrong. Theoretical analysis can only take you so far. But it can take you a lot further than we can go by guess-work, checking data patterns, or even looking at industry patterns.
Lesson 1: Cut through the joints, in other words, look for natural boundaries around business processes. As she points out, “if you get the natural boundaries right, it simplifies the number and complexity of interfaces. How do we do this? Melissa says “we need to take a gander at the enterprise goose, the whole thing.” A good check point is looking at data: “a ton of data elements or columns of data on an interface, you probably have the boundaries wrong.”
Lesson 2: Natural breakpoints or boundaries are similar for similar organizations. In other words, see what others that are in similar businesses have done and use this for a model.
Lesson 3: Boundaries don’t change based on size of the organization. Big turkeys carve much like little chickens.
I agree with Melissa strongly on two points. First, if you get your enterprise architecture right, you will make a great leap forward in managing the complexity of your systems. Second, your success in getting the architecture right is reflected in the data structures moving around your system.
Having said that, I feel several points need more comments. Let’s start with her thoughts on data. While I agree that poor enterprise architectures result in complex data patterns, I disagree that this is a good test to use to evaluate your enterprise architecture. The reason is simple. By the time you can notice the problems with data, it is far too late to do anything about the problems with the architecture. You have already implemented your systems and any changes in the enterprise architecture at this point are going to be very expensive.
So what is the solution? Any enterprise architecture should be tested against theoretical models for enterprise architectures based on the mathematical laws of complexity. These tests can be conducted BEFORE the architecture is implemented and can pinpoint problem areas before money is spent on their implementation at a time when they are still relatively inexpensive to fix.
Another area that we need to look at more closely is her lesson 2. While it is true that similar organizations have similar boundaries, this is not particularly helpful information. Why? Because “similar” is not good enough. Even small errors in “cutting up the chicken” can lead to huge problems with complexity. I personally have seen millions of dollars lost due to a single poor function placement. Again, the answer is to methodically check each function for correct placement, not based on a particular chef’s preferences, but based on the strict mathematics of synergies.
Don’t get me wrong. Theoretical analysis can only take you so far. But it can take you a lot further than we can go by guess-work, checking data patterns, or even looking at industry patterns.
Monday, November 10, 2008
The Job of the Enterprise Architect
ZDNet recently published an article on My Awesome IT Job: Enterprise Architect, IBM. The article is an interview with Martine Combes, an Enterprise Architect working with the CIO Organization of IBM in France. In the article, Martine says, "It is very easy to become overwhelmed by complexity and it is very difficult to maintain simplicity, especially when the scope is large, the number of team members is high. So better to have a simple architecture from the start, clearly showing the benefits for the Business."
On one hand, I applaud Martine's awareness of the importance of simplicity when it comes to Enterprise Architecture. On the other hand, I feel there is some degree of naivety in this understanding of the nature of complexity.
Marine appears to be saying that if you build your architectures simple from the beginning, you will have addressed the problem of complexity. There are several problems with this understanding of complexity. First of all, making an architecture "simple" depends on the eyes of the viewer. What is simple to me may be complex to you. A better goal than making architectures "simple" is to make architectures "as simple as possible." It is not possible to prove that an architecture is "simple", since "simple" is subjective. But it is possible to show that an architecture is "as simple as possible." The test for "as simple as possible" involves showing that the functionality follows the rules of synergistic partitioning. If you can find a subset of functionality that violates synergistic partitioning, then you know the architecture can be made simpler. If there is no such subset, the architecture is as simple as it can possibly get, and any further efforts to simplify the architecture will actually make it more complex.
My second issue with Martine's understanding of complexity involves the implicit assumption that just because an architecture starts simple that it will stay simple. In fact, architectures tend towards increasing complexity unless ongoing effort is put into keeping them simple. It is harder to keep an architecture simple than it is to make it simple in the first place. It requires an continuing analysis of new functionality and rigorous placement of all new functionality according to the rules of synergistic partitioning.
Whose job is it to keep make sure than architectures start out "as simple as possible" and stay that way for the system's lifespan? In my view, this is the primary responsibility of the enterprise architect. Only this person has the business perspective needed to identify synergies and the technical perspective needed to understand partitioning.
An organization that lacks a viable program in enterprise architecture will pay a severe cost in IT complexity. Complexity is the enemy. Enterprise Architecture is the solution. The only solution.
- Roger Sessions
On one hand, I applaud Martine's awareness of the importance of simplicity when it comes to Enterprise Architecture. On the other hand, I feel there is some degree of naivety in this understanding of the nature of complexity.
Marine appears to be saying that if you build your architectures simple from the beginning, you will have addressed the problem of complexity. There are several problems with this understanding of complexity. First of all, making an architecture "simple" depends on the eyes of the viewer. What is simple to me may be complex to you. A better goal than making architectures "simple" is to make architectures "as simple as possible." It is not possible to prove that an architecture is "simple", since "simple" is subjective. But it is possible to show that an architecture is "as simple as possible." The test for "as simple as possible" involves showing that the functionality follows the rules of synergistic partitioning. If you can find a subset of functionality that violates synergistic partitioning, then you know the architecture can be made simpler. If there is no such subset, the architecture is as simple as it can possibly get, and any further efforts to simplify the architecture will actually make it more complex.
My second issue with Martine's understanding of complexity involves the implicit assumption that just because an architecture starts simple that it will stay simple. In fact, architectures tend towards increasing complexity unless ongoing effort is put into keeping them simple. It is harder to keep an architecture simple than it is to make it simple in the first place. It requires an continuing analysis of new functionality and rigorous placement of all new functionality according to the rules of synergistic partitioning.
Whose job is it to keep make sure than architectures start out "as simple as possible" and stay that way for the system's lifespan? In my view, this is the primary responsibility of the enterprise architect. Only this person has the business perspective needed to identify synergies and the technical perspective needed to understand partitioning.
An organization that lacks a viable program in enterprise architecture will pay a severe cost in IT complexity. Complexity is the enemy. Enterprise Architecture is the solution. The only solution.
- Roger Sessions
Thursday, October 30, 2008
The Future of Enterprise Architecture
Kristian Hjort-Madsen recently posted a blog entry about the future of Enterprise Architecture in the government. Having done some work on architectural simplification for several public sector clients, this got me thinking about not only the future of EA in the government, but the future of EA in general.
It is my belief that we need to rethink Enterprise Architecture from being something that is focused on understanding the Enterprise to something that is focused on delivering value to specific projects. To some extent, this calls into question the name, "Enterprise Architecture", but I still think the name has value because it implies that we are considering project level issues that can only be addressed at the enterprise level.
To go even further, the main value that "Enterprise Architecture" can deliver to projects is in reducing their overall complexity. There are a number of secondary issues on which we can also focus, such as improving business/IT communications, but I believe that most of these will greatly benefit just from reduced project complexity.
Why do we need to focus on complexity at the Enterprise Level? Why not the IT level or the business level? The answer to this requires an overview of the Science of Simplicity. This is a bit much to cover here, but is the topic of my last book, Simple Architectures for Complex Enterprises and numerous white papers at my web site. In brief, simplification requires a good understanding of both synergistic relationships (which are best understood by the business side of the enterprise) and strong partition boundaries (which are best understood by IT side of the enterprise). We can thus only address these issues at the juncture point between business and IT, and that juncture point is what we usually call "Enterprise Architecture".
I think that government is perhaps the area that can particularly benefit from a focus on simplification. Government projects tend to be highly complex and often involve multiple vendors. When complexity is not managed methodically and early on, failure is too often the result. An excellent example of the problems inherent in such projects is the UK NHS system known as NPfIT (National Programme for Information Technology). This system will likely cost 50-100 billion dollars, and most likely will end up in failure. Why? Catastrophic complexity.
Why is complexity such a problem? Several studies (and numerous personal observations) have shown that increasing the functionality of a system by 25% increases the complexity of the system by 100%, unless steps are taken to manage the complexity. This means that taking a system from 10 functions to 100 functions increases the complexity by over 1000 times! Clearly this is not acceptable. The only way we can dampen this increase is by focusing on the problem of complexity. And Enterprise Architecture is where we have the opportunity to do so.
So let me summarize my thoughts on where Enterprise Architecture needs to go in the near term, especially in the public sector. First, it needs to move from an enterprise focus to a project focus. Second, it needs to narrow its focus to those problems that can only be solved from an enterprise (that is, a broad business/IT) perspective. Third, it needs to hone in on the one major problem that can only be solved from an enterprise perspective: complexity.
If we, as Enterprise Architects, make these adaptations, we will start making important, measurable project contributions and delivering highly visible short term value. And, in doing so, ensure our own long term survival.
It is my belief that we need to rethink Enterprise Architecture from being something that is focused on understanding the Enterprise to something that is focused on delivering value to specific projects. To some extent, this calls into question the name, "Enterprise Architecture", but I still think the name has value because it implies that we are considering project level issues that can only be addressed at the enterprise level.
To go even further, the main value that "Enterprise Architecture" can deliver to projects is in reducing their overall complexity. There are a number of secondary issues on which we can also focus, such as improving business/IT communications, but I believe that most of these will greatly benefit just from reduced project complexity.
Why do we need to focus on complexity at the Enterprise Level? Why not the IT level or the business level? The answer to this requires an overview of the Science of Simplicity. This is a bit much to cover here, but is the topic of my last book, Simple Architectures for Complex Enterprises and numerous white papers at my web site. In brief, simplification requires a good understanding of both synergistic relationships (which are best understood by the business side of the enterprise) and strong partition boundaries (which are best understood by IT side of the enterprise). We can thus only address these issues at the juncture point between business and IT, and that juncture point is what we usually call "Enterprise Architecture".
I think that government is perhaps the area that can particularly benefit from a focus on simplification. Government projects tend to be highly complex and often involve multiple vendors. When complexity is not managed methodically and early on, failure is too often the result. An excellent example of the problems inherent in such projects is the UK NHS system known as NPfIT (National Programme for Information Technology). This system will likely cost 50-100 billion dollars, and most likely will end up in failure. Why? Catastrophic complexity.
Why is complexity such a problem? Several studies (and numerous personal observations) have shown that increasing the functionality of a system by 25% increases the complexity of the system by 100%, unless steps are taken to manage the complexity. This means that taking a system from 10 functions to 100 functions increases the complexity by over 1000 times! Clearly this is not acceptable. The only way we can dampen this increase is by focusing on the problem of complexity. And Enterprise Architecture is where we have the opportunity to do so.
So let me summarize my thoughts on where Enterprise Architecture needs to go in the near term, especially in the public sector. First, it needs to move from an enterprise focus to a project focus. Second, it needs to narrow its focus to those problems that can only be solved from an enterprise (that is, a broad business/IT) perspective. Third, it needs to hone in on the one major problem that can only be solved from an enterprise perspective: complexity.
If we, as Enterprise Architects, make these adaptations, we will start making important, measurable project contributions and delivering highly visible short term value. And, in doing so, ensure our own long term survival.
Tuesday, October 21, 2008
Can an EA be "correct"?
In a recent blog, Nick Malik asserted that EA is not about building apps right, but is about building the right apps. And that the correctness of an EA was a point of view.
This was my response:
In my view, EA is not about either building apps right OR building the right apps. It is about creating a structured framework within which meaningful and collaborative dialogues can occur between the business and IT groups. The term that I use to describe this "structured framework" is Enterprise Architecture.
In most organizations, "meaningful and collaborative dialogues" between business and IT do not occur. You must ask why this is so. Is it because the business folks are overly demanding and unreasonable? Is it because the IT folks are unable to understand what business is saying?
In my view, the problem is neither the business nor IT. The problem is that EA has either not been done at all or, if it has been done, has been done incorrectly. Which brings us back to Nick's original point. Is there a "correct" way to do EA?
The answer is absolutely, yes. But the starting point needs to be an understanding of what we mean by "correct". I define "correct" as "as simple as possible". So of two EAs that both accurately define the enterprise, the more correct one is the simpler of the two. And the EA that is "absolutely" correct is the EA that is the simplest possible EA.
Why do I believe that simplicity is the raison d'etre for EA? The answer goes back to my original question. What is it that gets in the way of meaningful dialogues between business and IT? In my opinion, what gets in the way is complexity.
When IT blames the business for fuzzy requirements, IT is wrong. When the business blames IT for inability to deliver, the business is wrong. The problem is neither IT nor the business. The problem is complexity. Until complexity is solved, dialogue is impossible. Complexity is the enemy, and it is the common enemy of both IT and the business.
So can you determine if an EA is as simple as possible? Again, the answer is yes. But to do so requires both an understanding of the mathematics of complexity and a rational process for testing an EA for being "as simple as possible".
These are both topics I have covered extensively in my book, "Simple Architectures for Complex Enterprises", so if you are interested in these ideas, that is where to go. I also have a number of papers on this topic at my website, http://www.objectwatch.com/.
Best Wishes,
Roger Sessions
This was my response:
In my view, EA is not about either building apps right OR building the right apps. It is about creating a structured framework within which meaningful and collaborative dialogues can occur between the business and IT groups. The term that I use to describe this "structured framework" is Enterprise Architecture.
In most organizations, "meaningful and collaborative dialogues" between business and IT do not occur. You must ask why this is so. Is it because the business folks are overly demanding and unreasonable? Is it because the IT folks are unable to understand what business is saying?
In my view, the problem is neither the business nor IT. The problem is that EA has either not been done at all or, if it has been done, has been done incorrectly. Which brings us back to Nick's original point. Is there a "correct" way to do EA?
The answer is absolutely, yes. But the starting point needs to be an understanding of what we mean by "correct". I define "correct" as "as simple as possible". So of two EAs that both accurately define the enterprise, the more correct one is the simpler of the two. And the EA that is "absolutely" correct is the EA that is the simplest possible EA.
Why do I believe that simplicity is the raison d'etre for EA? The answer goes back to my original question. What is it that gets in the way of meaningful dialogues between business and IT? In my opinion, what gets in the way is complexity.
When IT blames the business for fuzzy requirements, IT is wrong. When the business blames IT for inability to deliver, the business is wrong. The problem is neither IT nor the business. The problem is complexity. Until complexity is solved, dialogue is impossible. Complexity is the enemy, and it is the common enemy of both IT and the business.
So can you determine if an EA is as simple as possible? Again, the answer is yes. But to do so requires both an understanding of the mathematics of complexity and a rational process for testing an EA for being "as simple as possible".
These are both topics I have covered extensively in my book, "Simple Architectures for Complex Enterprises", so if you are interested in these ideas, that is where to go. I also have a number of papers on this topic at my website, http://www.objectwatch.com/.
Best Wishes,
Roger Sessions
Thursday, August 21, 2008
Complexity is not a technology problem
In a recent copy of CIO Magazine, John Parkinson wrote an article titled Simplifying IT Management is Anything But. In this article, John discussed the difficulty in seeing through the vendor presentations to see which technologies would really make a difference to an organization.
In my view, if you are trying to understand what technologies to use to address complexity, you are probably too late. The solution to complexity is never found in technology. It is found in architecture. And architecture is mostly technology neutral. My perspective on complexity is that it can only be addressed at the enterprise architectural level, and then only if an organization specifically focuses on that as the overriding issue. In my book, Simple Architectures for Complex Enterprises, I address the issue of architectural complexity in depth. In summary, we must first understand complexity, then model it, then build processes that can be proven to address it. But the single most important factor is attitude. We need to view complexity as an enemy and simplicity as a core business asset. Every business analyst and IT architect needs to unite in the battle against complexity. Most IT systems fail, and the bigger they are, the harder they fall. When an IT system fails, the reason is almost always unmanaged complexity.
In my view, if you are trying to understand what technologies to use to address complexity, you are probably too late. The solution to complexity is never found in technology. It is found in architecture. And architecture is mostly technology neutral. My perspective on complexity is that it can only be addressed at the enterprise architectural level, and then only if an organization specifically focuses on that as the overriding issue. In my book, Simple Architectures for Complex Enterprises, I address the issue of architectural complexity in depth. In summary, we must first understand complexity, then model it, then build processes that can be proven to address it. But the single most important factor is attitude. We need to view complexity as an enemy and simplicity as a core business asset. Every business analyst and IT architect needs to unite in the battle against complexity. Most IT systems fail, and the bigger they are, the harder they fall. When an IT system fails, the reason is almost always unmanaged complexity.
Friday, August 15, 2008
Simple Architectures - The Book (Discussion)
Simple Architectures For Complex Enterprises is now available. You can order it at Amazon. This book introduces a new way of thinking about Enterprise Architecture. The focus of Enterprise Architecture should not be on efficient use of IT, improving alignment between IT and the business, or any of the other traditional concerns of Enterprise Architecture. Why? Because none of these are the real problem, they are merely symptoms of the problem. The real problem is enterprise complexity. This book discusses the nature of enterprise complexity, why complexity causes so many problems for IT, how complexity can be understood, and, most important, how complexity can be attacked.
Would you like to discuss some of the issues raised in Simple Architectures? This is as good as place as any!
Would you like to discuss some of the issues raised in Simple Architectures? This is as good as place as any!
Subscribe to:
Posts (Atom)
