Applying Earned Value Management to Software Intensive Programs

Many information technology projects have been declared too expensive, too late. and they often don’t work well. The application of appropriate technical and management techniques can significantly improve the current situation. The main causes of the growth of these large-scale programs can be attributed to various causes related to excessive promotion, immature technology, lack of corporate technology roadmaps, instability of requirements, ineffective acquisition strategy, pipelines unrealistic program foundation, inadequate systems engineering, and inadequate manpower. affairs. This article provides a brief summary of four processes to resolve these issues.

Establishment of a Process for the Definition of Requirements and Development of Technical, Cost and Schedule Baselines

We all realize the importance of a quality, motivated workforce, but even our best people can’t perform at their best when the process isn’t understood or isn’t working to its best. A well-defined process is critical to defining the requirements and completing the initial cost and schedule estimate. Proper use of Performance-Based Earned Value® (PBEV) provides integration of project technical scope, schedule, and cost objectives; and the establishment of a benchmark plan for performance measurement. Additionally, using an analytics application to project likely cost and schedule based on actual performance provides realistic projections of future performance. Project success can be aided by defining the best objectives, planning for resources and costs that are directly related to those objectives, objectively measuring achievement against the plan, identifying performance trends and problems as early as possible, and taking timely corrective action. share.

A ten-step process for generating and estimating program requirements is presented in the book “Software Sizing, Estimation and Risk Management” (Dan Galorath and Michael Evans, 2007). The 10 steps are:

1. Establish the estimated reach

2. Establish the technical basis, ground rules and assumptions

3. Collect data

4. Estimate and validate the size of the software

5. Prepare Baseline Estimates

6. Review, Verify and Validate Budget

7. Risk quantification and risk analysis

8. Generate a project plan

9. Document estimates and lessons learned

10. Monitoring of the project throughout the development

The key here is to establish a set of auditable and repeatable steps to establish the requirements and develop the baseline cost estimate and schedule.

Identification of critical software management metrics

That most big software programs get into trouble is a proven phenomenon. Therefore, selecting the right set of software metrics to track is critical to the success of the program. Practical Software Measurement (McGarry, Card, Jones; Addison-Wesley, 2002) identifies seven categories of information and expands these categories of information into measurable concepts and then forward-looking metrics.

For earned value purposes, the most effective software metrics are those that relate to product size, schedule, quality, and progress. For software-intensive programs, measures of quantity (for example, the number of lines of code completed) do not accurately reflect the quality aspects of the work done on the program or actual progress, since things like lines of code Completed do not capture things like integrate, test, etc.

Size is often measured as Lines of Source Code (SLOC) or Function Points and is used as a measure of size for quotes and for earned value using a percentage completion method. There are two critical problems with this approach. First, traditionally there has been a significant error in the estimation of SLOC. And, the number of lines of code completed doesn’t necessarily reflect overall quality or progress toward a performance goal. Therefore, any progress metric based solely on SLOC is highly volatile. Whether selecting SLOCs, function points, use cases, or some other sizing artifact, a careful process must be used to establish a credible sizing metric. It is recommended that in addition to tracking progress towards a goal, size growth is also tracked.

Schedule metrics and procedures that typically relate to completion milestones are also a common tracking metric. Sometimes these definitions of milestones and completion criteria lack quantifiable goals. Often an incremental build is released that does not incorporate all the planned functional requirements or a developer claims victory after only testing the nominal cases.

Progress metrics can be very difficult for large software programs. It is generally accepted that no software is delivered free of defects. Software engineers expected that new languages ​​and new processes would greatly reduce the number of delivered defects. However, this has not been the case. The software still ships with a significant number of defects. The physical and practical limitations of software testing (the only way to determine if a program will work is to write the code and run it) ensure that large programs will launch with undetected bugs. Therefore, the discovery and removal of defects is a key metric for evaluating program quality.

Application of performance-based earned value (PBEV)

Performance-Based Earned Value® (PBEV) is an enhancement to the Earned Value Management Systems (EVMS) standard. PBEV overcomes the deficiencies of the standard with respect to the measurement of technical performance and quality (quality gap). PBEV is based on standards and models for systems engineering, software engineering, and project management that emphasize quality. The hallmark of PBEV is its focus on customer requirements. PBEV provides principles and guidance for cost-effective processes that specify the most effective measures of cost, schedule, and product quality performance.

Program managers expect accurate reporting of integrated cost, schedule, and technical performance when the supplier’s EVMS procedure complies with the EVMS standard. However, EVM data will be reliable and accurate only if the following occurs:

o The indicated quality of the evolving product is measured.

o The correct baseline measures of technical performance are selected.

o Progress is objectively assessed.

The use of EVM also generates significant costs. However, if you are measuring things incorrectly or not measuring correctly, EVM can be more expensive to manage and can provide less management value.

Due to the quality gap in the EVMS standard, there is no guarantee that the reported earned value (EV) is based on product metrics and product quality evolution. First, the EVMS standard states that EV is a measure of the amount of work performed and that the quality and technical content of the work performed is controlled by other processes. A software manager must ensure that EV is also a measure of product quality and technical maturity of evolving work products rather than just the amount of work done. Second, the EVMS principles address only the scope of project work. EVMS ignores the product scope and product requirements. Third, the EVMS standard does not require precise and quantifiable measures of progress. It states that objective EV methods are preferred, but also states that (subjective) management evaluation can be used. In contrast, other standards specify an objective measurement. Fourth, EVM is perceived as a risk management tool. However, EVMS was not designed to manage risk and does not provide guidance on the subject.

PBEV is a set of principles and guidelines that specify the most effective measures of cost, schedule, and product quality performance. It has several features that distinguish it from traditional EVMS, augmenting EVMS with four additional principles and 16 additional guidelines.

PBEV complements traditional EVMS with best practices. Its principles and guidelines allow for true integration of project cost, schedule, and technical performance. The hallmark of PBEV is its focus on customer requirements. Product scope and product quality measures are incorporated into the project plan. Progress is measured against a plan to meet all customer requirements. Measuring the wrong things doesn’t dilute management’s focus. Consequently, management can take prompt corrective action on deviations that threaten customer satisfaction and the company’s business objectives.

Using an analytical process to project cost and schedule based on actual performance

Once the definition of the requirement is completed; the baseline cost and schedule have been established; the appropriate metrics have been selected; and have a PBEV system in place, the final challenge is to implement a process that quickly and accurately estimates final cost and schedule based on actual performance. This analysis is best accomplished using an analytical/parametric process. Galorath Incorporated calls this process Control SEER. The purpose of SEER Control is to provide an understanding of project progress so that appropriate corrective actions can be taken when project performance deviates significantly from plan. SEER Control provides a “dashboard” that includes a health and status indicator for the project related to: schedule variance, time variance, cost variance, size growth, and defect discovery and removal.

At the heart of SEER Control is the ability to forecast the bottom line of the project based on actual performance to date. One of the main goals of SEER Control is to provide adequate supporting documentation (charts and reports) to support the software project management process and meet the needs of interested parties.

Conclusion

Management of software-intensive programs should be based on establishing requirements, developing a reliable baseline estimate for cost and schedule, selecting effective software metrics, applying performance-based earned value (PBEV), and using analytical processes to project cost and schedule. based on actual performance.

Author’s Note: This article was written with contributions from Dan Galorath, CEO of Galorath Inc. and author of Software Sizing, Estimation, and Risk Management and Paul Solomon, co-author of the book Performance-Based Earned Value®.