Capital Asset Management Framework: 11. Performance Measurement

• 11.1 Introduction
• 11.2 Framework for Performance Management
  • 11.2.1 Performance Measures
  • 11.2.2 Monitoring, Measurement, Evaluation and Revision

---

11.1 Introduction

The Capital Asset Management Framework is based, in part, on the principle of strong accountability in a flexible and streamlined process. An integral part of this accountability involves performance measurement – the process by which program, service, project and/or asset outcomes are measured against intended objectives.

In capital asset management, agencies can use performance measurement to determine:

• if the processes used to plan and procure assets are effective and appropriate or whether they can be improved
• whether the intended project (asset performance) and service delivery outcomes are achieved
• whether accountable parties have effectively fulfilled their roles and responsibilities, and
• identify and implement corrective actions if intended objectives are not achieved

At the corporate level, performance measurement helps to ensure that:

• the agency is managing within capital-related fiscal targets from year to year
• proposed strategies and projects are being implemented from year to year, and
• broad asset management goals are being achieved (e.g. average age of capital stock, facility utilization rates, and maintenance expenditure targets)

At the program or project level, performance measurement can be used to determine:

• how effectively assets support service delivery objectives
• how well projects are managed, including whether they are delivered on scope, schedule and budget and whether risks have been effectively managed, and
• whether physical assets are meeting their technical performance objectives

The following chapter offers guidance on both corporate and project level performance measurement, including criteria for developing and using performance measures to support a culture of continuous improvement.

11.2 Framework for Performance Management

Performance measurement should be a component of every public agency’s accountability framework.  As illustrated below in Figure 11.2, that framework should include initial performance measures, and make allowance for their subsequent monitoring, measurement, evaluation and revision.

Figure 11.2

11.2.1 Performance Measures

Throughout the planning phase (i.e. in the development of agencies’ capital plans, business cases, project plans and all related contracts), agencies should establish performance measures and targets that are linked to broader strategic goals. Table 11.2.1 outlines some of the key attributes agencies should consider when establishing performance measures.

Table 11.2.1

Key Attributes for Performance Measures

Performance Measure Attribute	Description
Results Oriented	Performance measures and targets should be results-oriented to reflect the initiative’s measured or estimated consequences.
Comparative	Comparative information and benchmarks (e.g. efficiency ratios, data from similar jurisdictions) should be used where available.
Diverse and Balanced	Agencies should develop a mix of relevant output, outcome and efficiency measures – balanced to provide several different perspectives on service delivery and project outcomes. Performance targets should consider a variety of conditions and factors that can affect the achievement and public understanding of outputs/outcomes.
Stable	Selected performance measures should be reasonably stable to allow an examination of changes over time.
Realistic	Performance targets should be realistic and carefully selected to provide the appropriate incentives.
Able to withstand scrutiny	Performance measures should be suitable for external reporting and should be able to withstand scrutiny. They should be clear, meaningful, easy to understand and straightforward to interpret.

11.2.2 Monitoring, Measurement, Evaluation and Revision

• External and internal environments should be monitored to obtain information that may signal a need to re-evaluate the organization’s objectives or management systems

2. Measurement and reporting of results:

• Agencies should measure actual results and compare them to expected outcomes
• Any variances should be identified, along with their underlying causes, and
• Results should be reported to the appropriate internal or external authority

3. Evaluating results against expected outcomes:

• Results should be evaluated to ensure they accurately reflect performance over time.  They must clearly indicate if the intended outcomes were achieved

4. Results Management:

• This is the process by which appropriate action is taken to improve performance, based on results to date. Educated, informed reviews should be carried out to identify achievements, difficulties and needed corrections
• Results management helps to ensure that:
  • information gathered through performance measurement adds value to managing the overall directions of the program, agency, service and/or asset
  • results are used to assess the ongoing relevancy of the program, objectives and strategies, and
  • agencies strive for continuous improvement in critically reviewing results and taking corrective action – realigning strategies and performance measures as needed

11.2.2.1 Application to Capital Plans

Clearly-defined performance measurement criteria should be included in agencies’ capital plans to allow for an accurate assessment of their performance. These criteria would typically include:

• key corporate capital performance measures (e.g. targets for average age of the capital stock, track record for managing within capital-related fiscal targets)
• key project or program performance measures as identified through strategic options and business case analyses, and
• relevant benchmarks against which progress can be measured

Capital Asset Management Plans should also summarize agencies’ track records in meeting corporate and project-specific performance measures in previous years.

11.2.2.2 Application to Process Performance

All projects, whether they use alternative or traditional methods, should be subject to critical post-completion review and evaluation. Critical reviews of the project delivery process can identify positive and negative lessons learned and thereby inform future processes.

Process reviews typically focus on two phases of a project’s life cycle:

1. The planning, approval and procurement phase, from business case development and performance measure identification - through to the issuance of tender documents, RFEOIs and RFPS
2. The contract management and evaluation phase, including the project management and construction processes. For alternatively procured projects, reviews can be used to determine if the private partner is meeting performance targets within the contract’s parameters. They can also help determine whether the contract management resources and mechanisms in place are effective

11.2.2.3 Application to Asset Performance

Once a capital project is completed, post-implementation reviews (PIRs) can be used to determine whether its performance and business case objectives are being achieved.

Using surveys and other measurement methodologies, PIRs evaluate facility performance by focusing on users’ needs – including health, safety, security, functional and efficiency requirements, as well as user satisfaction.

PIRs are another tool that agencies can use to assemble “lessons learned” by collecting, archiving and sharing information about successes and failures in processes, products and other related areas, to help improve the quality of future facilities.  For example, PIR's can help agencies:

• fine tune asset specifications and performance levels
• assess facilities’ impacts on occupants or users
• measure facilities’ contributions to service outcomes
• gather information to adjust repetitive programs, and
• measure physical asset performance in terms of quality of design, construction and operating performance

A typical PIR has three components:

1. Scope and cost reviews, involve the evaluation of a completed asset’s technical scope and cost against the originally approved scope and budget
2. Program reviews, evaluate an asset’s functional performance in terms of user satisfaction and the original functional program
3. Product reviews, evaluate a completed asset’s physical performance in terms of design, specifications, life-cycle cost and operating and maintenance characteristics

11.2.2.4 When to Undertake a Process or Asset Review

Agencies should develop their own policies and processes for managing a performance measurement system. This should include establishing a framework and guidelines for conducting performance reviews of the types described above.

Often, the intention to conduct a review is identified in the business case and is part of project approval.

Generally, reviews should begin within six to 18 months of project completion – and should be completed for all high-risk and/or high-projects. For moderate-risk projects, agencies should evaluate a representative sample of initiatives over a given year. For low-risk projects, only a selective sample should be evaluated.

More specific criteria are outlined below in Table 11.2.2.4. Agencies should consider these criteria in selecting projects for process or asset reviews, and when developing internal guidelines and procedures.

Table 11.2.2.4

Specific Criteria for a Process or Asset Review

Criteria	Questions to Consider
Capital program or project	Is the project or program unique? For example, is it being delivered through a new or alternative approach? Or is it a repetitive program with which the agency has significant experience?
Process	Was the process used to acquire the asset and/or services unique? Did it present new or different risks (e.g. design-build, public-private partnership, construction management)?
Product	Is a new product or idea involved (e.g. alternative roofing material, new mechanical system, new or complex technology)?
Risk	Is the project high risk with probable unmitigated risks?
Costs	Is it high cost, with large capital or long-term operating impacts?
Impact Analysis	Does the project generate significant social, environmental or economic impacts?
Duration	How much time has elapsed since the agency last reviewed this particular type of asset or project?

---

10. Reporting & Monitoring < Previous | Next >  12. Financing