Para 8.4.3 — CONSULT_MANUAL

Rating Assessment Detailed Evaluation, in case of unquantifiable Criteria Marks A Very Good The consultants have outstanding, advanced expertise in specific problem areas of the assignment that can promise an excellent execution of the assignment. The consultants’ staff includes top experts in the field of the assignment. The consultants are considered world-class specialists in the approaches and methodologies dealing with specific issues in the assignment. The consultants operate according to well-established Quality Management (ISO 9002 etc.) Procedures. Full Marks B Good The consultants have extensive experience in the field of the assignment and have worked in Regions and Sectors with similar physical and institutional conditions, including similar critical issues. Permanent staff are adequate and highly qualified to cover the requirements of the assignment. The consultants have experience with advanced approaches and methodologies for dealing with the specific requirements of the assignment. 80% of full Marks C Satisfactory The consultants have experience in the field of assignments similar to the one being considered but have not dealt with critical issues specific to it (such as, for instance, delicate social or environmental issues). The consultants are experienced in the use of standard approaches and methodologies required for the assignment. The consultants’ permanent staff are adequate. 60% of full Marks D Unsatisfactory The consultant has experience which is not considered adequate for the quality needed by the Project. 30% of full Marks

2. Precise and exact markings of criteria and sub-criteria specified in technical evaluation (especially of unquantifiable criteria e.g., evaluation of Methodology) may neither be feasible nor warranted, especially when there is bound to be variation among marks by different members of CEC. Instead of assigning marks over the full range of attributes, it is more appropriate to divide the range into 4-5 slabs of ratings. A possible example of rating could be:

8.4.3 Rating/ Grading Schemes to Mitigate Subjectivity 1. Technical proposals for consultancy services are an intellectual product. Their evaluation must be based on individual professional judgement of competent evaluators and should not be reduced to a purely arithmetical exercise. The difficulty is to ensure that this judgement is not exercised in an unreasonable or arbitrary manner. It is important that subjectivity, implicit to any individual professional judgement, be complemented by transparency, consistency, and fairness. The individual evaluator entrusted with the evaluation, when required, should be able to explain to the satisfaction of a qualified reviewer from the higher authority or to enforcement agencies the reason for his/her scoring and recommendation. One way to achieve this objective is by adopting a rating/ grading scheme for evaluation of the criteria and sub-criteria (if so specified in the RfP) in the technical proposals.

Manual for Procurement of Consultancy Services, Second Edition, 2025

Rating Assessment Detailed Evaluation, in case of unquantifiable Criteria Marks E Not Relevant The consultant’ experience has no or little relevance to the Project under consideration. 10% of full Marks

3. Each member of the CEC should first read all proposals, without scoring them. This first review helps determine whether the proposals are free of significant omissions or deviations from the ToR; it also allows CEC members to assess the overall clarity of the proposals and identify elements that will require special attention in the evaluation.

4. After the review, the CEC meets to define the grades of the rating system to be adopted for scoring the technical proposals (if not detailed in the RfP), according to the criteria and sub criteria set out in the RfP. To discourage subjectivity and avoid the use of points and fractions of points, the rating system provides a few grades (from three to four) for each criterion and sub-criterion. Minimum qualifying marks or relative qualifying method for quality of the technical proposal will be prescribed and indicated in the RfP. The grading system must be defined before the technical proposals are opened to prevent bias (or perceived bias) occurring because of the CEC’s knowledge of the opened proposal contents. It is recommended that the evaluation and scoring of technical proposals be carried out only after defining the grading system. Otherwise, CEC members would have to assign a level of responsiveness of the proposals to each of the different criteria and sub criteria without guidance and support from predefined grades. This could easily distort the evaluation for the following main reasons:

a) Evaluators may differ, even widely, in their definition, understanding, or interpretation of the same criterion and also because of their subjective experience and understanding of the ToR; b) Disparities in evaluators’ relative generosity or severity in judgment and ratings can easily be magnified by the lack of common definitions of the requirements to be considered for each criterion and sub-criterion; c) Large differences in scores caused by inadequate understanding of the ToR or improper use of the evaluation criteria and sub-criteria are difficult to reconcile and explain.

5. After the rating/ grading scheme has been defined and proposals have been opened, the evaluation process can begin.