Why is it important to use fact-based decision making in software development?  Too many executives are prepared to treat software development as too complex to measure (or manage?). Instead, they operate in a state of continuous dissatisfaction based on occasional unsuccessful forays into the software development team which rarely yield significant change.  This is both uncomfortable and, potentially, illegal.  For example, as Davenport and Harris remind us in “Competing on Analytics,” the Sarbannes-Oxley Act of 2002 requires executives and other users of corporate data to demonstrate that their decisions are based on trustworthy, meaningful, authoritative and accurate data.  The act also requires executives to sign off that the data provides a clear picture of the business, major trends, risks and opportunities.

Can you or your colleagues really say this about your software development metrics?  Do you need help?

Consider the following 5 questions that you should ask yourself:

1. Is our software development data relevant?
2. Do we know where it came from (is it auditable)?
3. Do we have enough data (or too much)?
4. How can the data be made more accurate and valuable for analysis?
5. What rules and processes do we have in place to manage our software development metrics dat from its creation to its retirement?

“Val IT” is the 3rd in the series of four metrics for measuring the business value of IT that I introduced in my posting of July 7, 2009.  The first two in the series were the “Business Value Index” and “Total Economic Impact” (TEI).  Val IT is a complementary framework to the COBIT IT governance framework (described in our book, “The Business Value of IT” and elsewhere). While Val IT does not require a simultaneous COBIT implementation, it is clearly a good IT valuation option for organizations which already have COBIT in place.

Like TEI, Val IT started out focused on new IT investments.  Val IT is made up of three key processes which contain a total of 41 key management practices:

1. Value Governance (11 key management practices)

• Governance, Monitoring & Control
• Provision of strategic direction
• Define investment portfolio characteristics

2. Portfolio Management (15 key management practices)

• Identify and maintain resource profiles
• Define investment thresholds
• Evaluate, prioritize, select, defer or reject investments
• Manage the overall profile
• Monitor and report on portfolio performance

3. Investment Management (15 key management practices)

• Identify business requirements
• Develop clear understanding of candidate investment programs
• Analyze alternatives
• Definition and documentation of business case including benefits
• Assign accountability and ownership
• Manage through the economic life cycle
• Monitor and report on performance

“Total Economic Impact” is the 3rd in the series of postings on measuring the business value of IT that I introduced in my posting of July 7, 2009.  The 2nd in the series was the “Business Value Index” and TEI contains a number of characteristics that overlap with BVI:

• the use of a business case
• valuing intangibles
• calculating financial returns.

TEI adds:

• A methodology for quantifying risk
• The value of flexibility.

TEI has been used by Forrester for valuing IT investments.  It is of limited use in measuring the value delivered by IT in “normal operations” unless you take a view that the while IT budget is the investment. TEI is claimed to be more rigorous than BVI but the price of this is greater complexity.

TEI has four main components for assessing the total impact of a new investment:

1. Costs- the impact on IT

This looks at the costs incurred by IT compared to the costs of the status quo.  From the desired “Total Cost of Ownership” perspective, the “impact on IT” can be positive when money is saved or negative when money is spent.

2. Benefits - the impact on the business

Essentially, this is similar to the “Costs” analysis in that monetary flows, negative and positive, are aggregated, in this case, for all the non-IT departments of the organization.  Such flows would include improved productivity, training and so on.

3. Flexibility - Future Options

Derived from financial trading world, flexibility is measured as the value that this investment creates in terms of the ability to pursue other options in the future which are not currently available. The value of future options created is aggregated and added to the TEI,  For example, the investment being considered may be a change in software architecture which has a certain immediate value.  Additionally, at some point in the future, this change in software architecture could make it possible to switch to a cheaper hardware platform.  The switch to the less expensive hardware platform is not part of this project so the savings cannot be included in this TEI but there is some future value in creating the options and that value can be included in this TEI.

4. Risk

The first three types of analysis in TEI are additive.  The risk analysis reviews these the analysis of these three and generates an range of potential outcomes.  From this range, an expected or most likely value can be deduced and applied.  For example, if there is a 90% chance that the cost of training the business users will be $10k and a 10% chance that it will be $20k then the risk policy of the organization can be applied to make the risk adjusted cost $10k, or $20k or (10*0.9 + 20* 0.1) = $11k or some other value. Risk can be thought of as a multiplier of the sum of the first three to created “risk-adjusted costs” and “risk-adjusted benefits” to generate a “risk-adjusted ROI.”