Capital Asset Management Framework: 11. Performance Measurement
- 11.1 Introduction
- 11.2 Framework for Performance Management
The Capital Asset Management Framework is based, in part, on the principle of strong accountability in a flexible and streamlined process. An integral part of this accountability involves performance measurement – the process by which program, service, project and/or asset outcomes are measured against intended objectives.
In capital asset management, agencies can use performance measurement to determine:
- if the processes used to plan and procure assets are effective and appropriate or whether they can be improved
- whether the intended project (asset performance) and service delivery outcomes are achieved
- whether accountable parties have effectively fulfilled their roles and responsibilities, and
- identify and implement corrective actions if intended objectives are not achieved
At the corporate level, performance measurement helps to ensure that:
- the agency is managing within capital-related fiscal targets from year to year
- proposed strategies and projects are being implemented from year to year, and
- broad asset management goals are being achieved (e.g. average age of capital stock, facility utilization rates, and maintenance expenditure targets)
At the program or project level, performance measurement can be used to determine:
- how effectively assets support service delivery objectives
- how well projects are managed, including whether they are delivered on scope, schedule and budget and whether risks have been effectively managed, and
- whether physical assets are meeting their technical performance objectives
The following chapter offers guidance on both corporate and project level performance measurement, including criteria for developing and using performance measures to support a culture of continuous improvement.
Performance measurement should be a component of every public agency’s accountability framework. As illustrated below in Figure 11.2, that framework should include initial performance measures, and make allowance for their subsequent monitoring, measurement, evaluation and revision.
Throughout the planning phase (i.e. in the development of agencies’ capital plans, business cases, project plans and all related contracts), agencies should establish performance measures and targets that are linked to broader strategic goals. Table 11.2.1 outlines some of the key attributes agencies should consider when establishing performance measures.
|Performance Measure Attribute||Description|
|Results Oriented||Performance measures and targets should be results-oriented to reflect the initiative’s measured or estimated consequences.|
|Comparative||Comparative information and benchmarks (e.g. efficiency ratios, data from similar jurisdictions) should be used where available.|
|Diverse and Balanced||Agencies should develop a mix of relevant output, outcome and efficiency measures – balanced to provide several different perspectives on service delivery and project outcomes. Performance targets should consider a variety of conditions and factors that can affect the achievement and public understanding of outputs/outcomes.|
|Stable||Selected performance measures should be reasonably stable to allow an examination of changes over time.|
|Realistic||Performance targets should be realistic and carefully selected to provide the appropriate incentives.|
|Able to withstand scrutiny||Performance measures should be suitable for external reporting and should be able to withstand scrutiny. They should be clear, meaningful, easy to understand and straightforward to interpret.|
Agencies are encouraged to take a systematic approach to performance measurement at the corporate and project levels. A systematic approach involves four, interrelated activities:
- Performance monitoring:
- Performance should be monitored against the targets and indicators identified in the agency’s (or project’s) objectives and plans
- External and internal environments should be monitored to obtain information that may signal a need to re-evaluate the organization’s objectives or management systems
- Measurement and reporting of results:
- Agencies should measure actual results and compare them to expected outcomes
- Any variances should be identified, along with their underlying causes, and
- Results should be reported to the appropriate internal or external authority
- Evaluating results against expected outcomes:
- Results should be evaluated to ensure they accurately reflect performance over time. They must clearly indicate if the intended outcomes were achieved
- Results Management:
- This is the process by which appropriate action is taken to improve performance, based on results to date. Educated, informed reviews should be carried out to identify achievements, difficulties and needed corrections
- Results management helps to ensure that:
- information gathered through performance measurement adds value to managing the overall directions of the program, agency, service and/or asset
- results are used to assess the ongoing relevancy of the program, objectives and strategies, and
- agencies strive for continuous improvement in critically reviewing results and taking corrective action – realigning strategies and performance measures as needed
Clearly-defined performance measurement criteria should be included in agencies’ capital plans to allow for an accurate assessment of their performance. These criteria would typically include:
- key corporate capital performance measures (e.g. targets for average age of the capital stock, track record for managing within capital-related fiscal targets)
- key project or program performance measures as identified through strategic options and business case analyses, and
- relevant benchmarks against which progress can be measured
Capital Asset Management Plans should also summarize agencies’ track records in meeting corporate and project-specific performance measures in previous years.
All projects, whether they use alternative or traditional methods, should be subject to critical post-completion review and evaluation. Critical reviews of the project delivery process can identify positive and negative lessons learned and thereby inform future processes.
Process reviews typically focus on two phases of a project’s life cycle:
- The planning, approval and procurement phase, from business case development and performance measure identification - through to the issuance of tender documents, RFEOIs and RFPS
- The contract management and evaluation phase, including the project management and construction processes. For alternatively procured projects, reviews can be used to determine if the private partner is meeting performance targets within the contract’s parameters. They can also help determine whether the contract management resources and mechanisms in place are effective
Once a capital project is completed, post-implementation reviews (PIRs) can be used to determine whether its performance and business case objectives are being achieved.
Using surveys and other measurement methodologies, PIRs evaluate facility performance by focusing on users’ needs – including health, safety, security, functional and efficiency requirements, as well as user satisfaction.
PIRs are another tool that agencies can use to assemble “lessons learned” by collecting, archiving and sharing information about successes and failures in processes, products and other related areas, to help improve the quality of future facilities. For example, PIR's can help agencies:
- fine tune asset specifications and performance levels
- assess facilities’ impacts on occupants or users
- measure facilities’ contributions to service outcomes
- gather information to adjust repetitive programs, and
- measure physical asset performance in terms of quality of design, construction and operating performance
A typical PIR has three components:
- Scope and cost reviews, involve the evaluation of a completed asset’s technical scope and cost against the originally approved scope and budget
- Product reviews, evaluate a completed asset’s physical performance in terms of design, specifications, life-cycle cost and operating and maintenance characteristics
Agencies should develop their own policies and processes for managing a performance measurement system. This should include establishing a framework and guidelines for conducting performance reviews of the types described above.
Often, the intention to conduct a review is identified in the business case and is part of project approval.
Generally, reviews should begin within six to 18 months of project completion – and should be completed for all high-risk and/or high-projects. For moderate-risk projects, agencies should evaluate a representative sample of initiatives over a given year. For low-risk projects, only a selective sample should be evaluated.
More specific criteria are outlined below in Table 18.104.22.168. Agencies should consider these criteria in selecting projects for process or asset reviews, and when developing internal guidelines and procedures.
|Criteria||Questions to Consider|
|Capital program or project||Is the project or program unique? For example, is it being delivered through a new or alternative approach? Or is it a repetitive program with which the agency has significant experience?|
|Process||Was the process used to acquire the asset and/or services unique? Did it present new or different risks (e.g. design-build, public-private partnership, construction management)?|
|Product||Is a new product or idea involved (e.g. alternative roofing material, new mechanical system, new or complex technology)?|
|Risk||Is the project high risk with probable unmitigated risks?|
|Costs||Is it high cost, with large capital or long-term operating impacts?|
|Impact Analysis||Does the project generate significant social, environmental or economic impacts?|
|Duration||How much time has elapsed since the agency last reviewed this particular type of asset or project?|