Evaluation Teams

For those solicitations awarded based on the highest score(s), the evaluation team will collectively determine which of the proponent(s) / respondent(s) will be successful.  This can be an immense responsibility, particularly if the contract(s) is a high dollar value or of considerable public interest.

This page is specific to awarding based on the highest score; if the solicitation will be awarded based on price, refer to Submission Evaluations for information on how to evaluate bids received.

Who and how many should be on the evaluation team?  What are they responsible to do? What happens if someone on the team knows one of the proponents / respondents?  How much time will this take?  These are great questions that everyone on the team should know before they begin their evaluations, as answers may influence who is able to be on the team.

Note that some ministries may have developed processes and guidance that differ from the general guidance provided here.  Refer to the ministry links at the right, or contact the ministry’s Procurement Specialist.

 

Who Should be on the Evaluation Team

Evaluation teams should include at least three people, one of which must be from the ministry issuing the solicitation (see the Core Policy and Procedures Manual (CPPM), section 6.3.3.b.3).  The team may include: the program manager, a technical person, a financial person, a client representative and/or a procurement representative from the ministry or Procurement Services Branch. The number of people on the team will depend on the complexity of the project, and should include anyone who has sufficient and appropriate knowledge of the acquisition. However, if the team is too large it may take more time to complete the process; three to six members usually works best.

Evaluation teams should not include individuals who report to another team member.  If this is unavoidable, at least one neutral party (i.e. outside of the reporting relationship) should be part of the team to avoid the possibility or perception that the supervisor is making all the decisions on behalf of the team.

Refer to Conflicts of Interest and Bias below for more information on who should be on the evaluation team.

If specific expertise is needed that is not available within the ministry, contractors or staff from other ministries or government organizations may be included.  Any contractors on the evaluation team should sign a confidentiality agreement, and cannot be affiliated with any proponent / respondent.  Contact the ministry’s Procurement Specialist, Procurement Services Branch, or Legal Services Branch if a confidentiality agreement is required.

 

Evaluation Team Responsibilities

The evaluation team is responsible to:

  • keep the submissions, and any notes they might make relating to them, in a secure place where others will not have access to them;
  • refrain from discussing the submissions or disclosing their contents to anyone other than fellow team members;
  • document evaluation findings in sufficient detail to support the scores awarded;
  • retain copies of all documents in accordance with the requirements of the relevant legal and policy requirements;
  • keep all notes, discussions, and point ratings confidential and not disclose their substance or details to anyone outside of the team;
  • be aware of relevant requirements relating to confidentiality and document retention, including those arising under the Freedom of Information and Protection of Privacy Act and the Information Management Act;
  • evaluate the submissions in accordance with the evaluation criteria stated in the solicitation document;
  • evaluate submissions solely on information contained therein, and not on any previous knowledge of a proponent / respondent or its business; and
  • score submissions in accordance with the methodology and point breakdowns established in the solicitation.

A critical component of this process is treating all proponents / respondents the same.  This means that the same evaluators must evaluate the same information for each submission competing against each other.  Some team members may evaluate everything in submissions, whereas others may only be responsible for certain sections.  For example, someone with a financial background may be responsible for evaluating the price component, and/or to evaluate financial statements. In addition, ministries may ask for assistance from other ministry staff who are not evaluators but whose expertise and advice may be helpful, provided that such assistance is provided for all submissions.  In this case, the evaluation team is still responsible for assigning a score to submissions, which may consider the input received from other areas of the ministry.

Although the evaluation team is not required to be involved until the evaluations begin, the process will be much easier if the entire team was involved in drafting the solicitation and preparing the evaluation criteria.

Evaluators are expected to be available for the duration of the evaluation process.   The time needed for evaluations can vary widely, depending on the complexity of the solicitation.  It may take as little as 60 minutes to evaluate each submission, or as much as two days. However, for the average Request for Proposals, evaluators should plan for 1/2 a day per proposal for individual evaluations, and about three proposals per day for consensus evaluations.  Evaluators should consider their usual workloads and availability when committing to the team and determining dates for completing the evaluation process.

 

Conflicts of Interest and Bias

Conflicts must not exist between an employee's private interests and the discharge of their public service procurement process duties.  A conflict of interest occurs when an employee's private affairs or financial interests are in conflict, or could result in a perception of conflict, with the employee's duties or responsibilities in such a way that:

  • the employee's ability to act in the public interest could be impaired; or
  • the employee's action or conduct could undermine or comprise:
    • the public's confidence in the employee's ability to discharge work responsibilities; or
    • the trust that the public places in the public service.

Conflict of interest standards are contained within the BC Government employee standards of conduct document. There are additional procurement-specific standards of conduct for public service employees engaged in government procurement processes.

But how should this information be interpreted?  Some conflicts are very clear, whereas others may not be as obvious.  For example, the following are clear conflicts of interest:

  • A submission from a proponent / respondent with whom an evaluator has a financial interest (e.g. a family member’s business, the employer of a family member, a company where the evaluator owns significant shares, etc.).  An evaluator’s objectivity may be questioned if there is a perception of personal financial gain if this proponent / respondent is successful.  In this case, the evaluator should be excused from the team, and may be replaced with someone else.
  • A submission from an evaluator’s friend.  Even if the evaluator feels that they can be fair and unbiased, the perception will be that the team was not objective in evaluating proposals as one of the evaluators wanted to do a favour for their friend.  Again, this evaluator should be removed from the team and may be replaced with someone else.
  • A submission from a charitable or non-profit organization that an evaluator supports or is significantly involved.  The evaluator should be removed from the team, due to perception (or perhaps the reality) that the evaluator skewed results to favour this proponent / respondent.
  • A submission from a proponent / respondent who was involved in the development of the solicitation document.  In this case, the proponent / respondent will be seen as having information that was not available to all proponents / respondents.  Any vendor involved in the development of the solicitation should be informed of their ineligibility to participate as or in affiliation with a proponent / respondent, and any submissions received under these circumstances should be rejected.

Sometimes an evaluator will have a business relationship with one or more of the proponents / respondents, and may even have been named as a reference in their submission.  Is this also a conflict of interest?  Would this include the incumbent contractor, who has been delivering the services for the past few years?  How can the evaluation team be both knowledgeable about the services and not have worked with one or some of the proponents / respondents? 

If the relationship is strictly business (i.e. the evaluator and the proponent / respondent do not socialize outside of business, are not related or friends), there is usually no conflict of interest.  However, evaluators can only consider what is included in submissions when evaluating, meaning that anything additional known about a proponent / respondent cannot be used for scoring purposes.  Even if an evaluator is named as a reference, no conflict usually exists if the relationship is strictly business-related.  In this case, if references are checked the evaluator named in the submission should not be the one conducting the references, but rather would be one of the people contacted the same as with other references (see Using References for more information).

Whereas conflicts of interest are seen as one proponent / respondent having a potential unfair advantage in the competition, bias is seen as one or more proponents / respondents being at an unfair disadvantage.  In other words, sometimes a perception or reality exists that a ministry does not want a particular proponent / respondent to be successful.  Consider the following scenarios as examples:

1. The incumbent contractor and the ministry contract manager have had a difficult relationship, with disagreements about whether or not contract deliverables have been met.  The current contractor feels that the ministry manager may try to ensure that they are not successful in the solicitation.

Possible Solution:  Consider removing the ministry contract manager from the evaluation team.  This individual may be involved in the preparation of the solicitation document and the evaluation handbook, but perhaps should not be an evaluator.  If the knowledge and expertise gained through managing such contracts is critical for selecting the successful submission, consider asking colleagues responsible for similar contracts to replace the contract manager on the evaluation team.  By removing this individual from the evaluation process, a perception of bias is less likely if the incumbent contractor is not successful.

2. The incumbent contractor has been providing services for some years, and the ministry is satisfied with their work and/or a new contractor would mean significant transition work.  Some proponents may feel that their submissions will not be fairly evaluated as the ministry does not want or is not able to change contractors.

Possible Solution:  The solicitation should disclose everything known to the incumbent contractor that others should know when developing their submissions.  If it is not possible to disclose everything, consider either non-disclosure agreements (where proponents / respondents sign the non-disclosure agreement before they are provided with confidential material) or not considering those undisclosed elements of service delivery in the evaluation of submissions.  Ministries should also describe how they will deal with transition costs and time in the solicitation document.  However, considering transition costs in the evaluation process can be problematic; ministries are advised to contact the ministry’s Procurement SpecialistProcurement Services Branch, or Legal Services Branch for advice.

3. A vendor has delivered numerous submissions to the ministry over the past few years, and has never been successful.  The vendor feels that they have either been "blacklisted" for some reason, or that the ministry is not interested in working with new vendors.

Possible Solution:  Determine whether or not the vendor has participated in a debrief session in the previous procurement processes.  If not, be prepared to extend a direct invitation for a debrief meeting if they are again unsuccessful, as understanding how their submission was evaluated can greatly enhance their chances of future success.  If they have had a debrief in the past, it may be that their future submissions will improve as they learn more about what is expected.

More information on conflicts of interest can be found at Standards of Conduct and Relationships with Contractors.  

 

Training the Evaluation Team

Not all members of the evaluation team will have had recent experience evaluating submissions, and therefore they may be unsure of what to do.  A meeting to finalize the evaluation handbook and to train evaluators on what to expect can be very helpful.  Ideally, this meeting should occur shortly before the solicitation closes to provide some time to edit the evaluation handbook, if needed, but close enough to the evaluation process to remember the training.

Training should cover:

  • an overview of how to use the evaluation handbook;
  • the evaluation process;
  • conflicts of interest;
  • what can be considered when evaluating;
  • what to document;
  • how to score; and
  • who to contact with any questions.