Phil Laplante of Penn State is a prolific and respected writer on many software and project topics as well as being part of my local IT community.  In the July/August issue of the IEEE IT professional magazine, Phil steps back and writes a thoughtful and though provoking article that is perfect for contemplation on the beach.

In “Nexialism and the Law of Unintended Consequences,” (there’s a search term you won’t use very often) Phil suggests that to understand complex interactions that could impact our project risk we need to spend at least some time thinking in simple terms about the big picture.  Nexialism is defined as “the science of joining in an orderly fashion the knowledge of one field of learning with that of other fields.”

OK so that sounds a little grandiose and you might be wondering how it impacts your project risks.  Think of it this way, how many times in a failing project did somebody know that it was going wrong early but was never asked?  Is the test manager communicating (talking and listening) with the coding team lead (and vice versa).  What is the opinion of the DBA’s?  The performance test team?  Like it or not, these are different disciplines (or fields if you like) with different perspectives and, too often, we do not take care to “join their knowledge in an orderly fashion.”

Keith Baxter of De-RISK, based in the UK, has developed a simple but effective way of capturing and visualizing the impact of risk on a project estimate.  The approach, called Quality Based Costing (QBC),  and some examples are provided in the following paper on Keith’s website:

Quality Based Costing Paper

QBC is based on the responses to structured questions about each project component which seek to identify four costs for that component:

• Absolute Minimum
• Best guess/realistic estimate
• Contingency added
• Disaster scenario

The approach is then to apply Monte Carlo simulation to derive a probability distribution of the likely cost.  I particularly like an estimation process that includes Monte Carlo simulation because it forces the project manager and organization to make an informed choice about the risk they wish to take on each project.

The May 19, 2010 edition of Computerworld magazine contained a couple of articles about the growing evidence that one of the reactions to tightening budgets in software development has been the increased use of open source software and the associated  “Hidden Snags”

It is clear that one of the attractions of open source is the availability of source code which is critical for integrating open source into the enterprise but this brings problems of its own (“Integration Jams”).  I have written before about the benefits of static code analysis and I really want to stress the value of getting a one-off analysis done of any significant open source code that you bring into your enterprise.  Such a one-off analysis can be delivered as a service and does not require purchasing or learning a new tool.

Software development for software products shares all of the opportunities and challenges of software development in corporations with a few majors quirks all of its own.  One of these is the issue of managing a code line for a single product while accommodating (or not) variations for specific customers.  If you are working or have worked for a software product developer, you will be familiar with this problem.

Essentially, the easiest, cheapest and most efficient way to maintain a software product is to have a single code line with all clients on the same, latest version.  In this ideal model, any defects can be fixed in the single code base and provided to clients in the next maintenance release (maybe with a short-term patch to get over any short-term delay).  This model is illustrate below:

At the other extreme, different clients for a software product each have different versions of the software in production with differing levels of maintenance releases and patched loaded.  Many of the clients have their own special customizations to the code some of which overlap with the core product code. A simple example of the challenges is illustrated below:

Many software products will acknowledge more complicated scenarios than this.  Over the years, managing these scenarios has been made slightly easier by more sophisticated configuration management software which manages code branches and code line re-mergers.  That said, such systems still require a level of discipline comparable to the complexity they offer to manage and it is crucial to remember that some code branches just cannot be “automatically” re-merged.

If it is so difficult to manage code branches, why do software product vendors allow or even facilitate the branching of the core product code line?  The following drivers are all relevant:

1. New software product releases of any kind require effort from the clients to acceptance test before loading into  production.
2. A big license deal requires some updates to the product that are needed before the next release will be ready so the software product vendor creates a custom version for the “special”client.  When the next release comes out, the client is in the middle of configuration and implementation and does not want to take the new release.
3. There is pressure on software product vendors to provide minimal patch fixes to a clients specific problems to avoid the need to load maintenance releases which fix every clients problems.
4. All clients believe that they are unique and need changes to software products to reflect their specific operating methods (instead of changing their operating methods to suit the software).
5. All clients understand the long-term costs of customizing software products and so strive to stay on the core product line without necessarily loading every release (Why load a new release if WE don’t need the new functionality or bug fixes?).

“AIE” is the last in the series of four metrics for measuring the business value of IT that I introduced in my posting of July 7, 2009.  The first three in the series were the “Business Value Index,” “Total Economic Impact” (TEI) and “Val IT.”

AIE is considered to be the most rigorous of these four approaches but it has not gained widespread use.  With its mathematical, statistical and economic underpinnings, AIE provides investment decision-makers with a high degree of confidence in its results but there is a steep learning curve and it requires significant expertise.  Key features of AIE include:

• Rigorous “Unit of Measure” definitions even for such normally subjective elements as customer satisfaction.
• Systematic uncertainty analysis - quantification of the risk of an IT investment in such a way that it can be directly compared with a non-IT investment.
• Calculation of the economic value of information - AIE works on the logic that information reduces uncertainty, less uncertainty improves decisions, better decisions result in more effective actions and effective actions improve profit or mission results.  AIE includes a mathematical framework for calculating an pricing all this in dollars!
• IT investments as an investment portfolio - this is perhaps obvious from the previous three features but the rigor of the previous three features allows the IT investments to be analyzed alongside, and compared using, the same tools as those used extensively for non-IT investments.

As often happens, i was looking for something else when I came across a 2008 article in IEEE Computer by Mary Lacity and Joseph Rottman (don’t worry - the subject matter is covered in their 2008 book, “Offshore Outsourcing of IT work).  Having just set one of our clients off on the implementation of some outsourcing options to supplement their in-house resources, the list of 20 major effects of offshore outsourcing reported by project managers caught my eye.  I strongly recommend that you think of this list as a set of risks that need to be mitigated in any outsourcing implementation.  With apologies to the authors, I have sorted their list according to my highest priorities.  The in-house Project Managers reported that they:

• needed a mentor the first time they managed a project with offshore resources
• had to motivate the supplier to share bad news
• had to make offshore suppliers feel welcome and comfortable
• needed to thoroughly verify the offshore supplier’s work estimates, which tended to be optimistic
• had to provide greater detail in requirements definitions
• had to do more knowledge transfer up front
• were forced to shortcut the knowledge transfer process because of deadlines set by senior IT leaders
• had to ensure that knowledge transfer was successful by testing the supplier employees’ knowledge
• had to set more frequent milestones
• needed more frequent and more detailed status reports
• required more frequent working meetings to prevent client-caused bottlenecks
• needed to accompany offshore suppliers to all client-facing meetings
• experienced higher transaction costs which threatened their ability to deliver projects on budget
• experienced project delays which threatened their ability to deliver projects on time
• had to guarantee that the supplier followed pre-agreed knowledge renewal practices
• had to ensure that the supplier transferred knowledge about new applications or technologies to the client
• had to learn about new applications or technologies independent of suppliers to ensure that the suppliers information and bids were valid
• had to integrate the suppliers CMM/CMMI processes into their own project management processes
• had to ensure that the supplier’s employees were fully trained as promised by the suppliers
• had to fill many of the roles the the PMO should have performed

I came across an interesting article in the May/June 2009 issue of IEEE Software by Magne Jurgensen of the Simula Research Laboratory in Norway which touches on the age-old challenge of deciding whether the lowest estimate is actually a realistic estimate.

From his research, Magne has derived 7 recommendations which I have paraphrased below (I suggest they have much validity for internal and external estimates):

1. If price is an important selection criteria, dont invite too many bidders if your requirements specification is incomplete.
2. Avoid using price as an important selection criteria if your ability to assess provider competence is low.
3. Ensure sufficient competence in assessing provider competence by hiring external experts if necessary.
4. When selecting bidders, compare the bids with your own independent cost estimate of the average of all bids.
5. Let providers collect sufficient information to help them analyze the projects complexity.
6. Don’t include potentially misleading information that could affect the estimation process (especially your budget expectations).
7. Be aware that a negotiation phase after receipt of bids may encourage even greater over-optimism.

Of course, “over-optimism” is Magne’s polite way of saying “stupidity.”  If these recommendations resonate with you, I suggest that you should talk to us here at DCG about getting a professional, objective view of your estimates and/or your estimation process.

There are a number of issues associated with allowing your staff to use Free and Open Source Software (FOSS).  There are more issues associated with allowing your software developers to incorporate FOSS in your projects and applications especially if you are in the business of reselling such software as part of a product or service.

High on the list of challenges in both cases are the license implications.  Equally high on the list must be concerns about the code quality of the FOSS.  When it comes to validating the quality of FOSS, there is a simple strategy that could allow you to have more confidence in FOSS than in the off-the shelf software packages that your organization probably uses.

How?  Well, the nice thing about FOSS is that you actually have access to the source code.  This means that you can run it through a source code analyzer such as those available from CAST Software and their competitors.  You can reasonably expect that a good source code analyzer will not only report on the efficiency and quality of the code but also on any potential security threats.  Using an analyzer in this way will provide a layer of certainty that you do not have over the packaged or COTS software you buy without access to the source code.

The business case?  The time saved in bringing in FOSS solutions over the time taken to write and evolve them yourself should cover the cost of the analysis tool.  If you use a lot of FOSS then it makes sense to invest in buying the analysis software yourself.  If you use FOSS only occasionally then DCG can help you by performing one-off analysis as a service using the CAST Software tool whenever you need it.

“Total Economic Impact” is the 3rd in the series of postings on measuring the business value of IT that I introduced in my posting of July 7, 2009.  The 2nd in the series was the “Business Value Index” and TEI contains a number of characteristics that overlap with BVI:

• the use of a business case
• valuing intangibles
• calculating financial returns.

TEI adds:

• A methodology for quantifying risk
• The value of flexibility.

TEI has been used by Forrester for valuing IT investments.  It is of limited use in measuring the value delivered by IT in “normal operations” unless you take a view that the while IT budget is the investment. TEI is claimed to be more rigorous than BVI but the price of this is greater complexity.

TEI has four main components for assessing the total impact of a new investment:

1. Costs- the impact on IT

This looks at the costs incurred by IT compared to the costs of the status quo.  From the desired “Total Cost of Ownership” perspective, the “impact on IT” can be positive when money is saved or negative when money is spent.

2. Benefits - the impact on the business

Essentially, this is similar to the “Costs” analysis in that monetary flows, negative and positive, are aggregated, in this case, for all the non-IT departments of the organization.  Such flows would include improved productivity, training and so on.

3. Flexibility - Future Options

Derived from financial trading world, flexibility is measured as the value that this investment creates in terms of the ability to pursue other options in the future which are not currently available. The value of future options created is aggregated and added to the TEI,  For example, the investment being considered may be a change in software architecture which has a certain immediate value.  Additionally, at some point in the future, this change in software architecture could make it possible to switch to a cheaper hardware platform.  The switch to the less expensive hardware platform is not part of this project so the savings cannot be included in this TEI but there is some future value in creating the options and that value can be included in this TEI.

4. Risk

The first three types of analysis in TEI are additive.  The risk analysis reviews these the analysis of these three and generates an range of potential outcomes.  From this range, an expected or most likely value can be deduced and applied.  For example, if there is a 90% chance that the cost of training the business users will be $10k and a 10% chance that it will be $20k then the risk policy of the organization can be applied to make the risk adjusted cost $10k, or $20k or (10*0.9 + 20* 0.1) = $11k or some other value. Risk can be thought of as a multiplier of the sum of the first three to created “risk-adjusted costs” and “risk-adjusted benefits” to generate a “risk-adjusted ROI.”

Eileen Feretic, writing in the June 2009 edition of Baseline magazine, reported a recent study by the Hackett Group which forecasts an 8.6% growth in IT demand between now and 2011 with only a 1.3% increase in budget.

Surprised?  No, I thought not. But do you have a plan?  Are you comfortable that your plan, if it exists, can deliver the implied 7.3 cost reduction?

If you answered “No” to either of these questions, where should you start?

The answer is simple, you have to start withe the business.  If you don’t have one already, you need to develop an IT Governance methodology (DCG has one and it can help you with this if desired) that identifies all the important decisions at the business-IT interface, who makes them and how they are made.  This is essential for getting the inevitable prioritization exercise right.  Too often cost saving is arbitrary and causes disproportionate harm to wither the business or IT.

Feretic’s article quotes Steve Bozz, CIO of 1-800-Flowers, “What we’ve done is take all the tech stuff and translate it into business language so that its meaningful to our brands and business units.  Now they can make decisions in a way they never could before.”