Post Contract Evaluation

Post contract evaluation is more than best practice. Section 6.3.6.c.3 of the Core Policy and Procedures Manual states that a post contract evaluation is required for all contracts valued over $50,000.

Some ministries have developed their own post contract evaluation templates and/or guidance; refer to the links at the right for this information, or contact the ministry’s Procurement Specialist.

Typically, post contract evaluations use source information (as noted below) to document the success of the contract that can then be used to evaluate three areas:

  1. Deliverables and Outcomes;
  2. Contractor Performance; and
  3. Internal Contract Management Team.

Post contract evaluations are used to determine how the ministry benefited from the contract, if all the deliverables were met, how well the contractor performed during the contract, and how well each member of the Province’s team performed his/her role.  These evaluations are documented to serve as a reference on the project and to identify any lessons learned.

Evaluations can:

  • Confirm the results that were anticipated in planning, and those indicated during the progress monitoring;
  • Help formulate future contracts that may require improved processes, and/or more detailed terms and specifications;
  • Create a final record of the contractor's performance; and
  • Assess and improve the performance of the internal contract management team.

Post contract evaluation processes should include a mechanism that (i) documents the subject’s (i.e. the contractor or the internal contract management team) agreement with the evaluation results, and (ii) allows them to object if they don’t agree.

Source Information

A complete contract file is important to a successful and useful evaluation. Source information that can help with post contract evaluations includes, but is not limited to:

  • Planning documents (needs assessment, feasibility studies, cost benefit justifications, and/or business cases that outline objectives, critical success factors, timelines, budget, deliverables, risks, and performance measures – see Plan for more information);
  • Progress/monitoring reports delivered by the contractor (see Manage for more information);
  • Documentation related to inspections, site visits, outside reviews, complaints, compliments;
  • Documentation on project deliverables, including the condition and timeliness of delivery;
  • Invoices (see Contract Payment for more information);
  • Notes from meetings with the contractor, the Province’s internal team and any other stakeholders;
  • Correspondence with the contractor and the internal team;
  • The contract, including any amendments (see Award for more information); and
  • If applicable, the original solicitation document and the contractor’s submission to it (see Pre-Award for more information).

In addition, some contract managers prefer to keep a file on issues encountered for the post contract evaluation as they arise during the procurement life cycle.  This methodology assists with the final compilation of a comprehensive evaluation.  At the other end of the spectrum, some contract managers prefer to rely on their memory.  The duration of the procurement can often be the deciding factor in which method is chosen.

Gather as much of this information as possible prior to conducting the post contract evaluation.

 

1. Deliverables and Outcomes

Post contract evaluations include confirming that all the deliverables were met, to the quality standards and quantity identified in the contract.  When determining whether or not the deliverables were met, refer to what was actually stated in the contract rather than what should have been stated.

The expected contract deliverables should have been received as per the specifications of the contract. However, it may be that the deliverables were fully met, but that the project or program overall did not meet its objectives.  Therefore, the broader success of the contract should be evaluated in terms of the project or program of which it was a part.

When conducting a post-contract review where the objectives of the project or program were not met, the ministry should determine which of the following reasons likely contributed to this failure:

  • There was no clear connection between the objectives and the deliverables (meaning that successful completion of the deliverables was not likely to result in the desired outcome);
  • The deliverables were not well defined, and therefore the ministry did not receive what was expected; and/or
  • The contractor did not provide the deliverables as stated in the contract.

Whether or not the overall objectives were met, the original specifications should be reviewed to determine if any problems encountered with deliverables during the contract term could have been prevented.  This review will help to determine what changes would be required if a similar contract were needed in the future.  Questions such as, "Were the deliverables completed according to the stated standards/specifications?", “Were the deliverables clear and understood by all parties?”, and "Did ministry staff receive any new ideas or approaches that can now be used for future contracts of this kind?" prove to be very valuable to the contract process as a whole.

To conduct an evaluation of deliverables:

  • Gather all the applicable source information;
  • Either as a group or individually:
    • determine whether or not the overall objectives, outcomes or purpose of the contract was achieved, and
    • review the results of the contract compared to the original expectations;
  • Document results (using either the template below or one developed for the ministry – see the ministry links to the right);
  • Share results of the evaluation with management (if appropriate); and
  • Include the review in the contract file.

The results of contracts can generally be evaluated, but some may be difficult to directly link to outcomes.  For instance, it is not always possible to clearly attribute improved client satisfaction, savings, increased output volume, improved controls or higher productivity to any one factor.   However, logical assumptions can be made based on the facts available to determine whether or not the contract contributed to a positive outcome.

Those ministries that do not have their own template may want to use the corporate template to evaluate deliverables.

 

2. Contractor Performance

Evaluating the contractor's performance at the end of the contract is critical to the procurement life cycle, as it helps to continually improve the contract process.  Reputable, professional vendors recognize that their credibility and future contracting assignments depend on their contract performance.

The ministry should identify how well the contractor performed the works/services and how well they worked with ministry staff and any other outside agency or stakeholder.  Did the contractor satisfy its obligations under the contract?  Was the contract completed within the original, or a revised and approved, schedule and budget?  If not, why not?

Performance standards for the contract may have been set out in the planning process, included in the original solicitation, and/or defined in the contract documents.  However, contractors can only be held accountable for those performance standards that were included in the contract; if the contract did not contain any or the standards were not clearly articulated, any performance issues encountered may be a result of a misunderstanding rather than a contractor’s inability or unwillingness to meet expectations.  This distinction is critically important, particularly if the performance evaluation may inform future reference checks and recommendations on whether or not to contract with this vendor again.

The content of performance evaluations should be substantiated by appropriate documentation such as the contractor’s work deliverables, a record of communications between the ministry's representatives and the contractor's representative, and any other relevant contract record pertaining to the quality and/or quantity of the work.

To conduct an evaluation of contractor performance:

  • Gather all applicable source information;
  • Either as a group or individually, review the contractor performance as it relates to the standards described in the contract;
  • Document results (using either the template below or one developed for the ministry – see the ministry links at the right);
  • Share results of the evaluation with the contractor, and provide the contractor an opportunity to comment;
  • Share results of the evaluation with management (if appropriate); and
  • Include the review in the contract file.

Those ministries that do not have their own template may want to use the corporate template to evaluate contractor performance.

 

3.  Internal Contract Management Team

Most people learn something new with each contracting experience that they have. In fact, sometimes the best learning opportunities come when dealing with the unexpected.  Therefore, it is useful to document how the internal team performed – particularly when faced with something unexpected – in order to apply these learnings to future contract opportunities.  Another purpose of evaluating the internal team is personal growth, by identifying any staff training needs, and planning for future mentoring opportunities that will enhance staff learning.

The evaluation plan for the contract management team should be discussed during the planning stage.  Each of the team members can be assigned specific roles and responsibilities for the procurement life cycle, and their performance on how well they managed should be evaluated.  The members of the internal team may also bring different ideas about what should be evaluated based on experiences in other contracts.  Questions such as, "Should more time have been spent on establishing the specifications?", "Did team members reply promptly to emails and telephone calls?" and "Could the team do a better job next time?" all provide valuable information to contract managers for their next procurement.

Using the applicable source information, there are many ways to conduct a review of the internal contract management team:

  • Each team member can individually document how well the team performed anonymously, using either a ministry template (see the links to the right) or the one found below;
  • Team performance can be documented jointly as a group; or
  • Team performance can be documented by the contract manager alone.

Whatever the approach, the review should be included in the contract file. Those ministries that do not have their own template may want to use the corporate template to evaluate the internal contract management team.