Software Measurement Consulting

Performance Benchmarking

In the drive for efficiency and profitability IT professionals will find it helpful to benchmark their organization’s performance against its peers.

DCG will work with you to develop a performance benchmark to measure your critical success factors such as software quality, time to market and cost reduction. Increase the value of your benchmark analysis and include Industry Benchmark data representing Industry Best Practices.

"I often say that when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is a meagre and unsatisfactory kind: it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be." - Lord Kelvin, the 19th century British scientist who"invented" Absolute Zero (among other things).

BENCHMARKING PRODUCTIVITY AND QUALITY PERFORMANCE

To achieve effective Benchmarking Productivity and Quality Performance DCG follows a rigorous three step process:

Performance

• Selection of Projects
• Functional Sizing
• Levels of Performance

Capacity

• Process Capability
• Technology Effectiveness
• Recommendations

Comparison

• Internal Benchmarks
• Industry Benchmarks
• Best Practices

Performance
The Performance step of the benchmarking process includes the quantifiable measurement of productivity and quality levels.

Project Selection
Performance is determined through a representative sampling of projects selected by your organization. Data is collected (when available) on the size of the deliverable, level of effort, project duration and number of defects. These measures are analyzed, and performance levels are established.

The selection of projects is commonly based upon your unique criteria:

• Completed or in development during last 18 months,
• Include all business areas,
• Greater than 6 staff months to complete,
• Representative of future projects,
• Include all platforms,
• Mix of technologies and
• Include all languages.

Projects meeting these criteria are further reviewed to determine the availability of data. The resulting list includes representative projects based upon technology, size and business unit.

Functional Sizing
Each selected project is sized using Function Points. The function point methodology provides for the evaluation of a software deliverable based on the amount of functionality delivered. Sophisticated formulas and objective analysis of system characteristics are part of this industry-accepted sizing practice.

Other data including effort and schedule duration is collected from project managers. If the data is not readily available, it is recreated through rigorous examination of project activities that occurred during the life of the project.

Levels of Performance
Performance levels are computed and graphically displayed. These graphs depict individual project performance levels relative to organizational benchmarked average levels of performance.

Capacity
Capacity represents the qualitative portion of the Benchmarking process.

Process Capability & Technology Effectiveness
A capacity analysis reveals the influence of current software practices upon performance levels. From this assessment, it is possible to recommend improvements in your current practices, suggest new practices and emphasize existing practices that have already demonstrated positive influences on your productivity.

Data is collected on each selected project. The method of collection involves interview/survey sessions with your project teams. Data is collected on six key influence factors: project management, definition, design, build, test and environment.

Individual projects are analyzed, and project profiles are created. This data is also analyzed in aggregate and used as the basis for determining recommendations.

Recommendations
Recommendations are developed which focus on those activities that have the best opportunity for the greatest impact on the productive development of software in your environment. They are also focused at an organization level. The benchmarking process examines productivity rates and performance variables on a project-by-project basis. Based upon that data, a benchmark profile is created that is a composite picture of the organization. This provides an organizational profile of performance supported by the quantitative baseline performance levels. If the organization continues to do business similar to current patterns, it can anticipate that productivity will remain at the same level.
Therefore, the recommendations focus on the common characteristics that can be modified to enhance your current organizational profile. Five or six key variables are identified, which when changed organization-wide, will positively impact future projects.

Comparison

Internal & Industry Benchmarks
The comparison phase graphically depicts the current internal benchmark results and contrasts those results to industry benchmark data. Common displays include Productivity, Level of Effort and Schedule Duration; see the graphical examples below.

Best Practices
When these results are contrasted to the capacity profiles, the analysis reveals and quantifies your organization’s best software practices. In addition, identification and quantification of process strengths and weaknesses can be effectively benchmarked and used as a point of comparison as improvement initiatives are developed and executed.