While the full CRASH report must be purchased, considerable information is available in the free summary available here. I will highlight some of the most surprising, and in some cases, controversial findings. To make this more interesting, I will deliver my discussion as a quiz. So get ready!
CAST did this analysis using their software analysis tools. CAST produces tools that analyze software systems and rates them on various criteria of code quality. CAST, for example, can analyze the 50,000 lines of code that was just delivered from your outsourcing firm and rate it on maintainability, adaptability, security, and a number of other attributes that you probably care a great deal about.
The logic to doing this analysis is simple. Sooner or later, you are going to find out about how maintainable, adaptable, and secure this code is. Would you prefer to find out now, before you have accepted delivery, or later, after you have deployed this system to your trusting constituents?
To produce the CRASH report, CAST used their tools on a large collection of software systems ranging in size from 10K LOC (lines of code) to more than 5M LOC. They did this for a number of industries, programming systems, and development methodologies.
Okay, are you ready to take the CRASH quiz? Here goes! I will start with the questions and then give you the answers.
QUIZ
Q 1. Rank the following languages from most used to least used: ABAP, C, Java, .NET, Cobol.
Q 2. In the Government, which of the following languages is most popular: ABAP, C, Java, .NET, Oracle Forms?
Q3. Which of the following yield the worst security scores: Cobol, C++, or Java?
Q4. Which of the following have the highest complexity scores: Java, Oracle Forms, or Cobol?
Q5. Which industry has the highest complexity scores: Government, Financial Services, or Telecom?
Q6. Which code has a higher overall quality index, code produced in-house or code that is outsourced?
Q7. Which development approach produces code with a higher quality index, Agile/Iterative or Waterfall?
Q8. What is the "Technical Debt" in an average system, per line of code?
a. Less than $1.00 per line of code.
b. Between $1.01 and $2.00 per line of code.
c. Between $2.01 and $3.00 per line of code.
d. Between $3.01and $4.00 per line of code.
ANSWERS (No Peeking!)
A1. Overall, these languages rank (from most to least popular) Java, Cobol, ABAP, .NET, and C. Actually, C is rarely used. I include in the list just for nostalgia.
A2. In the Government, Oracle Forms is the most popular programming system, followed by Java. I note that Oracle Forms tends to be relatively small programs if one can even call them programs. So while Java is used for fewer "systems" I suspect (but can't tell from the data) that it is used for many more lines of code.
A3: From a security perspective, C++ and Java are in a virtual tie for worst security. Cobol code overall has a much better security score. This may reflect more on the industry than the language, since Cobol is used heavily in the Financial Services industry, where security is taken more seriously than, say Telecom where Java use predominates.
A4. The most complex code by far is found in the Cobol systems followed by Oracle Forms. Java wins for the least complex code of the three.
A5. The industry with the highest complexity scores is the Government. Financial Services is a distant second followed by Telecom. This result is surprising given than Java is popular in both Telecom and the Government. The implication seems to be that although the Government is using very good language tools, it is not maximizing their effectiveness.
A6. Code that is produced in-house has a better quality index than outsourced code, but the difference is marginal and probably not statistically significant.
A7. Overall, Waterfall development has a significantly higher quality index than does code produced using Agile/Iterative. Not only does Agile/Iterative score lower in overall quality, it also scores lower in transferability (the ability for other groups to understand the code) and changeability (the ability to modify the code.) I can hear the groans of protest already from the Agile community. Sorry, I'm just the messenger.
A8. The average "Technical Debt" in a system is $3.61 per line of code (answer d.) The technical debt looks at the number of problems, the severity of those problems, and the cost of fixing those problems.
Some of these answers are a bit surprising, aren't they? Feel free to read the summary report here. You will probably find another surprise or two.
1 comment:
It's probably not surprising that waterfall produces less complex code, because it also produces the least suitable (fit for purpose). After some time in production the waterfall code will get complexer than the agile code. Or so do I like to believe...
Post a Comment