Past work experience can be a key evaluation element to any solicitation process. But what assurances are there that the information provided in the submission is accurate? Or that the work cited was completed on time and on budget? References can help.
- References and the Solicitation Document
- Process for Checking References
- Province as a Reference
- Contracts Valued at $10 Million or More
If asking for references, describe what is required and how the references will be used in the solicitation process. When developing the solicitation document, consider the following:
- The number of references required;
- Whether references will always be checked, or only at the discretion of the ministry;
- How references will be used; and
- Whether the ministry wants the option of checking additional references.
Number of References
When asking for references, consider the time it takes to contact all that were requested and the expected benefits of doing so. For example, if the solicitation asks for three references for the bidder / proponent / respondent organization plus three references for each of the two key personnel named, each submission will include nine references. Are that many references truly needed? How likely is it that the ministry will be checking all nine references? If more than just the highest scoring bidder’s / proponent’s / respondent’s references will be checked, consider how much time it will take to check all nine references from each of two or three highest scoring submissions. Consider asking for fewer references who can provide information on the critical experience needed.
Always Check References?
Generally speaking, having the option to check references, rather than always checking, works better. Consider what would happen if the lead submission was from a bidder / proponent / respondent that is known to perform well. If the solicitation document gave the option to check references, the ministry would not be obligated to do so as the outcome of those checks can be anticipated. However, if the lead submission was from an unknown bidder / proponent / respondent or the evaluation team had concerns with the experience cited, then references should be checked.
How to Use References
For those solicitations based on highest scores, assigning points to references is not generally useful. Usually, points assigned to references are a small fraction of the overall points available for experience. Therefore, even if the references are unsatisfactory, the impact on the overall score is minimal.
To illustrate what can happen if references are scored, consider the following example:
A Short-form Request for Proposals (SRFP) is issued where proponent experience is worth 30 out of an overall total of 100 points. 25 of those 30 points are allocated to the experience cited in the proposal, and the remaining 5 points are specific to the references.
Proponent A submits a very polished proposal that is specific to everything requested in the SRFP and that demonstrates an in-depth level of relevant experience. Before the references are checked, they scored 23 for their experience. However, when the ministry checks their references, all describe concerns with the quality of the services provided, and problems with the proponent’s ability to complete the previous projects on time and on budget. One reference even stated that the delays and inferior quality control resulted in a decision to hire another vendor to complete the work. Based on this, references are awarded 1 point for an overall total of 24 out of 30 for proponent A’s experience.
Proponent B submits a proposal that lacks the specific details requested in the SRFP. Although it appears that this proponent also has in-depth and relevant experience, details are lacking to fully understand what they have done. This proposal scores 17 for experience, before the references are checked. When the ministry checks their references, all give glowing reviews of the wonderful work this proponent did for them, and how they would hire them again without hesitation for similar work. Proponent B is awarded all 5 points for experience, for a total of 22 out of 30 for experience.
In this example, even though the references are a clear indication of who is likely to perform better, proponent A is scoring higher in experience than proponent B.
Instead of assigning points, consider a pass/fail process, where references have no bearing on scores but the ministry can exclude a proponent / respondent from consideration because of poor references. If the references for the highest-scoring proponent / respondent are acceptable, this approach saves time as no further reference checks would be required. However, if the references for the highest-scoring proponent / respondent are problematic, DO NOT exclude the proponent / respondent from consideration until after consulting with Legal Services Branch, as the legal risks can be significant.
The pass/fail approach to references can apply to those solicitations that are awarded based on lowest price.
The solicitation document should be clear on whose references will be checked. For example, the solicitation may state that only the lowest-priced or highest-ranked bidder’s / proponent’s / respondent’s references will be checked. Alternately, the solicitation could state that references from the highest scoring submission and all submissions scoring within x points of the highest will be checked.
Option for Additional References
Solicitations can include the option to check additional references. This can be very useful, particularly if the bidder / proponent / respondent cited experience in their submission where no reference was provided. With this option, the Province could search for contact information and include these other projects as part of the reference check process. If considering this approach, be sure to include language in the solicitation stating that the Province may check references in addition to those provided.
Once the consensus evaluations are complete for the submissions and the decision is made to check references, identify who will conduct these checks. This should be a single person for all references being checked, whether only one or multiple submissions’ references are included. This person could be one of the evaluators or someone else.
Reference checks should take place prior to announcing the results of the solicitation. Reference checks can be made via telephone or email. The appropriate questions to ask on a reference check depend on the nature of the services being purchased. These questions should be developed in advance and asked of all the references.
If references of all highest scoring proponents/respondents will be checked (i.e.more than one submission’s references), be sure to follow the same process for each. For example, if only one of the references provided will be checked for each proponent / respondent, use the same methodology to choose that one reference (e.g. the reference associated with the experience most closely aligned to the required services).
The results of reference checks should be documented well. If checks are made via email, the emails themselves are sufficient. If checks are made via telephone, record in detail what the references said, as well as the date and time of the call.
Some organizations have internal policies that do not allow them to do more than confirm whether or not a vendor has had a contract with them in the past. This is not a reflection – positive or negative – on the experience of the applicable bidder / proponent / respondent, and should not be considered as a failed reference.
Having an employee of the Province named as a reference has its benefits. It enables the evaluation team to consider their own or another ministry's experience with the bidder / proponent / respondent as part of the selection process.
However, negative references from the Province should be handled carefully. Negative references can be used effectively only if the Province has evidence in writing (e.g. emails, memos, letters) that the bidder / proponent / respondent:
- Was aware of the performance issue with the past contract;
- Had the opportunity to reply to and/or rectify the situation; and
- Did not rectify the situation in accordance to what was agreed upon.
If no written documentation exists demonstrating the above bullets, a negative reference from the Province should not be given. Telephone calls and meetings cannot be considered written documentation, unless an email or other written documentation describing the meeting or call exists that was shared with the bidder / proponent / respondent. If in doubt about how to respond when asked to be reference, contact the ministry’s Procurement Specialist or Legal Services.
Note that sometimes, an evaluation team member may be named as a reference in a submission. This is not necessarily a problem, provided that:
- The named evaluator is not in a conflict of interest with that bidder / proponent / respondent (i.e. the relationship is strictly business, not personal or financial - refer to Evaluation Teams for more information);
- The named evaluator does not use any knowledge that he/she may have of the bidder / proponent / respondent that is not included in the submission when evaluating; and
- The named evaluator is not the person checking references, if references are checked.
Unless the ministry has a policy or practice not to be a reference, any evaluation team member who is named as a reference should be treated the same as any other reference.
However, being named as a reference is not the same as providing a letter of recommendation or support. A reference is simply the name and contact information of someone who can verify the experience cited in a submission. A letter of recommendation or support is a document that endorses the vendor as providing quality services. Provincial staff usually can be named as a reference, but should not be providing letters of recommendation or support to vendors. Providing such a letter may give the perception that the Province is recommending that a particular vendor be awarded a contract solely based on past experience with the Province rather than the applicable solicitation process. This perception may exist even if the letter is used for a solicitation issued by another ministry or government entity.
Refer to the Core Policy and Procedures Manual, section 6.3.3.b (1) Vendor Reference Check Review Policy for information on the reference check expectations for contracts valued at $10 million or more. This includes a pre-qualification process with an internal performance reference check component.