Being able to estimate software development more efficiently and effectively is both possible and practical. The following key points will guide you to a successful outcome.

•    We need to recognize estimating as a problem and a potentially costly problem at that. Not until we fully understand that improper estimating is a potential barrier will we be able to consistently and successfully deliver software.

•    The way we think about estimating should be reframed in the context of managing expectations based upon the best information available at the time. Estimating is not a crystal ball used to predict the future.

•    We can no longer afford to compromise on what we need regarding the input components that make up a successful estimating model. It will require some investment of time and resources but the payback will be well worth the investment.

•    And above all, don’t overly complicate your estimating model. Adopt practices such as FP Lite to generate size information that is statistically accurate enough for the job of early estimating. Collect the baseline data you need and compute your own internal delivery rates of performance.

Why is it important to use fact-based decision making in software development?  Too many executives are prepared to treat software development as too complex to measure (or manage?). Instead, they operate in a state of continuous dissatisfaction based on occasional unsuccessful forays into the software development team which rarely yield significant change.  This is both uncomfortable and, potentially, illegal.  For example, as Davenport and Harris remind us in “Competing on Analytics,” the Sarbannes-Oxley Act of 2002 requires executives and other users of corporate data to demonstrate that their decisions are based on trustworthy, meaningful, authoritative and accurate data.  The act also requires executives to sign off that the data provides a clear picture of the business, major trends, risks and opportunities.

Can you or your colleagues really say this about your software development metrics?  Do you need help?

Consider the following 5 questions that you should ask yourself:

1. Is our software development data relevant?
2. Do we know where it came from (is it auditable)?
3. Do we have enough data (or too much)?
4. How can the data be made more accurate and valuable for analysis?
5. What rules and processes do we have in place to manage our software development metrics dat from its creation to its retirement?

“Val IT” is the 3rd in the series of four metrics for measuring the business value of IT that I introduced in my posting of July 7, 2009.  The first two in the series were the “Business Value Index” and “Total Economic Impact” (TEI).  Val IT is a complementary framework to the COBIT IT governance framework (described in our book, “The Business Value of IT” and elsewhere). While Val IT does not require a simultaneous COBIT implementation, it is clearly a good IT valuation option for organizations which already have COBIT in place.

Like TEI, Val IT started out focused on new IT investments.  Val IT is made up of three key processes which contain a total of 41 key management practices:

1. Value Governance (11 key management practices)

• Governance, Monitoring & Control
• Provision of strategic direction
• Define investment portfolio characteristics

2. Portfolio Management (15 key management practices)

• Identify and maintain resource profiles
• Define investment thresholds
• Evaluate, prioritize, select, defer or reject investments
• Manage the overall profile
• Monitor and report on portfolio performance

3. Investment Management (15 key management practices)

• Identify business requirements
• Develop clear understanding of candidate investment programs
• Analyze alternatives
• Definition and documentation of business case including benefits
• Assign accountability and ownership
• Manage through the economic life cycle
• Monitor and report on performance

“Total Economic Impact” is the 3rd in the series of postings on measuring the business value of IT that I introduced in my posting of July 7, 2009.  The 2nd in the series was the “Business Value Index” and TEI contains a number of characteristics that overlap with BVI:

• the use of a business case
• valuing intangibles
• calculating financial returns.

TEI adds:

• A methodology for quantifying risk
• The value of flexibility.

TEI has been used by Forrester for valuing IT investments.  It is of limited use in measuring the value delivered by IT in “normal operations” unless you take a view that the while IT budget is the investment. TEI is claimed to be more rigorous than BVI but the price of this is greater complexity.

TEI has four main components for assessing the total impact of a new investment:

1. Costs- the impact on IT

This looks at the costs incurred by IT compared to the costs of the status quo.  From the desired “Total Cost of Ownership” perspective, the “impact on IT” can be positive when money is saved or negative when money is spent.

2. Benefits - the impact on the business

Essentially, this is similar to the “Costs” analysis in that monetary flows, negative and positive, are aggregated, in this case, for all the non-IT departments of the organization.  Such flows would include improved productivity, training and so on.

3. Flexibility - Future Options

Derived from financial trading world, flexibility is measured as the value that this investment creates in terms of the ability to pursue other options in the future which are not currently available. The value of future options created is aggregated and added to the TEI,  For example, the investment being considered may be a change in software architecture which has a certain immediate value.  Additionally, at some point in the future, this change in software architecture could make it possible to switch to a cheaper hardware platform.  The switch to the less expensive hardware platform is not part of this project so the savings cannot be included in this TEI but there is some future value in creating the options and that value can be included in this TEI.

4. Risk

The first three types of analysis in TEI are additive.  The risk analysis reviews these the analysis of these three and generates an range of potential outcomes.  From this range, an expected or most likely value can be deduced and applied.  For example, if there is a 90% chance that the cost of training the business users will be $10k and a 10% chance that it will be $20k then the risk policy of the organization can be applied to make the risk adjusted cost $10k, or $20k or (10*0.9 + 20* 0.1) = $11k or some other value. Risk can be thought of as a multiplier of the sum of the first three to created “risk-adjusted costs” and “risk-adjusted benefits” to generate a “risk-adjusted ROI.”

At DCG, when we work with clients we often conclude that spreadsheets may be the most appropriate way to manage a simple metrics program.  However, I was reminded this week (based on an incident at a client which caused me to look up the appropriate section of “Competing on Analytics” by Thomas H. Davenport and  Jeanne G. Harris) that there are a couple of common problems with basing your metrics program on spreadsheet repositories:

• Errors - Raymond Panko wrote a much referenced article back in 1998 based on his research which suggested that 20-40% of user-created spreadsheets contain errors.  The more spreadsheets, the more errors.
• Multiple versions of the truth - too often there is not a single, multi-user spreadsheet but multiple versions being spread like a virus across an organization by email, each with its own few unique data values.
• Unintended uses - the data in spreadsheets are very difficult to keep under control when key data elements are changed to meet new needs.  The knock-on effect on linked data can be catastrophic but not necessarily visible.

Worst of all, its almost impossible to debug or do a data integrity check on even a mildly complex spreadsheet.

The lesson - use spreadsheets as much and as often as you like but do not attempt to build a serious metrics or estimating application on them if you want it to have a useful life in your organization of more than one year!

In my posting of November 18, 2009 entitled “DCG’s Software Development Value Index - A metric for effectiveness, efficiency and economy,” I referenced the concept that the system represented by a software development group takes inputs and performs a transformation of those inputs into outputs.  I proposed a system-based metric for assessing the value provided by the software development group as a result.

Now I would like to come back the concept that the transformation part of the process can be value neutral, value add or value multiplier.  What do I mean by those terms and how should they be prioritized when considering a project portfolio for a software development group?

A “Value neutral” project does not change perceived value of product or application e.g. some architectural changes, bug fixes, make work.

A “Value add” project clearly adds to perceived value but probably only for a subset of all the possible beneficiaries.  Examples might be interfaces to 3rd party software, incremental functionality to existing component, new local capability.

A “Value multiplier” project increases value for all possible beneficiaries.  Examples might be new front end, new component to enable new clients/market, new general capability, new language support.

Obviously, in an ideal world, all projects in a software development portfolio would be value multipliers.  However, that is not realistic and could even be a high risk allocation of resources.  In practice, a typical portfolio will contain a mix of the three types of project.

So what is the right balance?  It depends on the risk tolerance of the business, the available discretionary funds and the degree to which the business can drive the software development group to be flexible.  However, the DCG “rule of thumb” is to appotion your software development dollars somewhere between $1-$2-$1 and $1-$2-3 for value neutral - value add - value multiplier.  In other words, always try to spend twice as much on value add as value neutral then try to spend one to three times the value neutral amount of value multipliers.

Interesting article by Jeanne Harris, Allen Alter and Michael Nieves in the September 2009 issue of CIO Insight (OK - so my reading list got out of sequence!).  Jeanne is a co-author of “Competing on Analytics” which I liked a lot.

Anyway, the article sets out a plan for CIO’s to increase IT capability while cutting costs.  The report takes the broader view of IT (e.g. it includes operational costs such as “telecom expense”) and groups the priority actions by the quarter in which payoffs can be expected.  I was interested in where cost savings in software development could be achieved.  The answer seems to be that there are no direct cost savings to be had although there may be implied cost savings depending on how your IT is structured.

The list is as follows:

Returns in Q1 (”Minimize” phase):
SLA’s
telecom expense
hidden IT

Returns in Q2 (”Optimize” phase):
thin-client computing
procurement management
application renewal
consolidation and virtualization

Returns in Q3 (”Redesign” phase):
process engineering
shared services
operating model and organizational redesign
sourcing strategies

Which of these do you think are cost saving strategies around software development?  Let’s set aside that they have omitted the most obvious and usual Q1 step - lay off x% of the development staff.  It seems to be that the software development gains come from Q3 in the form of process improvements and sourcing strategies.  That resonates with my experience if, and only if, you start those initiatives immediately.

I believe in process and spent my early working years on the factory floor (managing a high-tech production line).  Consequently, its quite comfortable for me to fall into the common trap of wondering why we can’t just build software the way we build cars - on a production line - maybe even using robots for some of the tasks!.  From a process, measurement and statistical control perspective, the production line model would surely help us develop software better, faster and cheaper?

I was sitting in a conference last week listening to yet another speaker fall into this trap.

Why is it a trap?

Because before you can apply the production line model to software development (and complain about the shortcomings of software development groups), you need to do a little imagination experiment with me.

First of all, I need to you to conjure up that standard production line video in your minds for me - you know the one - with  shiny car chassis steadily moving down the line while humans and robots attach parts, paint parts, test parts, etc.  Pick your favorite car and imagine it being assembled.

With me?  OK, now I need you to consider that every car will be slightly different - some will have moon roofs, some will have two doors, some four and so on.  Well OK, these modern plants are very flexible and they can cope with a lot of that but the sophistication of the logistical process that supports the build has certainly jumped up a notch along with the industrial design engineering needed to make the production line work (quite separate from the design of the car!).

Hmm, what would be the equivalent to that sophisticated logistics and industrial engineering in software development groups.  Well, in most SD groups, we have some tools to help us with configuration management and testing (if we are lucky), probably some code libraries and maybe even some formal processes.

OK, lets get back to our mental picture of that sophisticated car production line.

Now I want you to imagine that the same line that produces your favorite car with trucks flowing down it.  And buses.  And Formula 1 cars.  And tanks.

I know what you are thinking - don’t be stupid, Mike.  You wouldn’t put all of those things down the same production line.  It wouldn’t be cost effective.  Some of those things will never be on a production line - even if such a production line could be designed.

Duh!

There are several approaches and sets of rules for counting Function Points (e.g. IFPUG, COSMIC, Mk II, etc.).  They are all designed to use that most powerful of processors, the human brain.  Designing a computer program to automate function point counting has proved impossible to date (and maybe forever) for two main reasons: the current rules assume the pattern recognition and variable input capabilities of the human brain.  Put more simply, the rules assume that the counter can receive information (in a wide variety of formats including subject matter interviews) and pick out relevant architectural information.

However,  software has evolved for other uses -mainly static code analysis - which uses algorithms to understand the structure of code to evaluate its quality from a code design and implementation perspective.  Clearly, such code analyzers can do a good job of measuring software size in the form of lines of code but as we all know that is not particularly useful.

Some vendors are now modifying these algorithms to develop software size metrics based on the underlying architecture and using the sort of rules that an IFPUG function point counter would use.  It is not realistic to expect these tools to exactly replicate IFPUG function point counts in all circumstances because the IFPUG rules (embodied in the IFPUG Counting Practices Manual) are simply not written in a way that allows a program to encapsulate them all (and the input format variability problem remains).  However, these tools are now exhibiting significant signs of being a disruptive technology in the Christensen sense in that while they may not be great at doing everything that a human function point counter can do, they may be better (cheaper) at doing some things e.g. a monthly or quarterly recount of an application portfolio.

Here at DCG, I am committing us to work with vendors of these new tools to see if we can develop them together to improve the overall software sizing services that we can deliver to users. For a longer explanation of where we are, please see the presentation that I made to the 2009 UKSMA Conference in London.

One of the ideas that I float in the presentation is the possibility of creating a different, new software size metric that can be automated - a sort of Automated Function Point.  A good client of ours, Thierry Fraudet, thought about this and wondered how it might work for COTS or SaaS software?  True to form, Thierry answered his own question by suggesting that users could require vendors of COTS or Saas to provide function point counts with their software.  What a great idea?  And why limit it to automated function points?  Why not start asking COTS and SaaS vendors for IFPUG function point counts now?  As Thierry says, “What an interesting way to compare license fees!”

If you are interested in automated function points, I would love to hear from you.

Is software development delivering value to the organization?

Taking a systems view of the software development organization, software development delivers value when it is:

• Effective => An output metric, O
• Efficient => A transformation metric, T
• Economical => An input metric, I

Software Development Value System

Here at DCG, we have developed the concept of a balanced Software Development Value Index (SDVI):

(x * I) + (y * T) + (z * O)

where priority weights x+y+z = 100%.

The answer to the annual or quarterly question, “Is the Software Development organization delivering value?” is “Yes” if the SDVI exceeds the annual target.

For example, for Year 20xx, SDVI priorities (or weights) are chosen by the Software Development (or IT) Governance Committee to be:

• Effective,  z=50%
• Efficient,  y=20%
• Economical,  x=30%

Let’s choose a target for value delivered of SDVI > 85%.

The Software Development Input, Output and Transformation scores are generated by combining and normalizing the scores for a set of metrics measured under each heading e.g. Input metrics might include budget and hours.

Lets assume that for our example software development group in Q1, 20xx the normalized metrics are:

• Effective, O = 87%
• Efficient, T = 90%
• Economical, I = 80%

Then our SDVI = (0.30*0.80)+(0.20*0.90)+(0.50*0.87) = 0.855
At 85.5%, we can conclude that software development delivered value in the first quarter of 20xx!

You will see that we have defined the concepts of “Value Neutral”, “Value Add” and “Value Multiplier” for our software development transformation process.  The meaning of these will be described in a future blog but essentially they refer to the transformation value of individual projects and are intended to help with the project prioritization process that is part of software development governance especially at budget planning time.  By the way, if you don’t have a Software Development Governance Committee, we should probably talk!