Unconscious bias in graduate admissions and scholarship adjudication

Role of the referee: To provide an assessment of the potential for a candidate to succeed in a graduate program, plan of research, or academic award.

Suggested strategies to minimize implicit bias when writing a letter of reference or other assessment of a candidate:

1. Avoid using stereotypical or interpersonal attributes when describing character and skills (e.g., words such as nice, hardworking, conscientious, dependable, diligent, kind, agreeable, sympathetic, compassionate, selfless, giving, caring, warm, nurturing, maternal);
2. Focus on research skills and achievements using words that describe the candidate’s research excellence (e.g., thought-provoking, innovative, novel, thorough, detailed, impactful);
3. Consider using, where appropriate, ‘stand-out’ adjectives (e.g., superb, excellent, outstanding, confident, successful, ambitious, knowledgeable, intellectual) for all genders (e.g., women, men, transgender, two-spirit);
4. Use the nominee’s formal title and last name instead of their first name. Avoid attributing the contribution of an applicant’s work to the order of authors, as not all disciplines follow a single convention;
5. Consider whether your comment unintentionally includes ‘doubt raisers’ (i.e., negative language, hedges, unexplained comments, faint praise and irrelevancies, such as “might make an excellent leader” versus “is an established leader”);
6. Use inclusive language is (e.g. “the applicant” or “they” instead of “he/she”) and do not use words or sentences that reflect prejudiced, stereotyped or discriminatory language of particular people or groups, or their institution;
7. Avoid being unduly personal;
8.  Support your points by providing specific examples of accomplishments always;
9. Do not include information related to ethnicity, age, hobbies, marital status, religion, disability;
10. Avoid sharing personal information about the applicant; such information may be helpful only in explaining academic delays or interruptions and should be addressed with caution.

Source: Adapted from the Vanier Graduate Scholarships website. (Canada, 2020)

Role of the review: To assess a candidate’s potential success in a program of study or academic award, and select candidates for recommended offers of admission or academic awards.

Suggested strategies to minimize implicit bias in the review and selection process:

1. Use clearly established, standardized assessment metrics and processes with a common rubric and an understanding of bias.
2. Allow sufficient time and avoid multi-tasking when you review applications to allow for self-correction of bias-related tendencies.
3. Review the selection criteria before you begin evaluating applications.
4. Question whether your evaluation would change if the applicant were of a different gender or cultural background, if they had a different name or if they were working on a different research topic.
5. Guard against over-reliance on one piece of information or on “first impression” reactions.
6. Organize the applications by random sorting the order of review numerically.
7. Pre-score all nominations assigned and submit pre-scores to the designated committee chair or administrator prior to the selection committee meeting.
8. Consult guidelines for reviewing nominations involving Indigenous scholarship or research with Indigenous communities, such as those available through your institution or Government of Canada.
9. Programs and committees can develop a standardized letter of recommendation with cognitive and non-cognitive scales.
10. Read the writing sample first, then look at test scores, then look to the letters for — ideally — concrete information that contextualizes the rest of the material in the packet, particularly the writing sample or candidate statement.
11. Pay attention to the facts within the specific content.
12. Establish the weight of each component of the assessment. (Letter of reference, grade point average, personal statement, writing sample relative to each other.)

Source: Adapted from the Vanier Graduate Scholarships website. (Canada, 2020)

What criteria are established predictors of success, through what indicators can these be assessed, and what sources of evidence relate in order to inform the assessment?

Basic example:

Your rubric can be more robust, sophisticated, and complete. This is just a simple illustration. See also Vanier Criteria for a more robust example.

Once you have established the Criteria, Indicators, and Sources, you may also want to determine the relative weight of each criterion category, and develop a scoring system where each candidate application receives a numeric score in each category and an overall total score.