In recent years, artificial intelligence (AI) has been making significant strides in various sectors, and higher education is no exception. A groundbreaking study by a PhD student at the University of Pennsylvania, Benjamin Lira Luttges, and a team of nine other researchers, demonstrates how AI can play a pivotal role in diversifying college campuses. The study, “Using Artificial Intelligence to Assess Personal Qualities in College Admissions,” published in Science Advances, sheds light on how AI algorithms can evaluate personal qualities such as pro-social orientation and leadership in college application essays. This development comes at a critical juncture, as traditional standardized tests like the SAT and ACT faced disruptions due to the COVID-19 pandemic, leaving college admissions officers to rely more heavily on application essays.
A response to the SAT and ACT dilemma
The COVID-19 pandemic disrupted the standard college admissions process, suspending standardized tests like the SAT and ACT. With these tests sidelined, admissions officers were compelled to place greater emphasis on high school grades and application essays. However, assessing applicants’ personal qualities, such as teamwork, leadership, and pro-social orientation, through essays presented its challenges. These essays were time-consuming to review and subjected to human biases and variations. Like anyone else, Admissions officers could be influenced by factors like fatigue or time of day when assessing applications. Recognizing these challenges, Benjamin Lira Luttges and his team explored how AI could aid in the evaluation process.
The role of admission essays
College admission essays have long been a tool for applicants to demonstrate their suitability for higher education institutions. These essays allow students to narrate their life experiences and explain why they are an ideal fit for a particular college or university. However, they are also fraught with subjectivity, as evaluating these essays often relies on the qualitative judgment of experienced admissions officers. The potential for bias in this process is a concern, as admissions essays can influence the composition of a college’s student body.
Interestingly, the Supreme Court’s recognition of the role of these essays in understanding applicants’ experiences and perspectives was emphasized in a ruling on affirmative action in college admissions. Chief Justice John Roberts stated that universities could consider an applicant’s discussion of how race affected their life. This recognition underscores the importance of gaining insights into applicants’ personal qualities through essays in a constitutionally acceptable manner.
The AI solution: RoBERTa
Luttges and his team’s solution to the challenge of assessing personal qualities in admission essays involved the use of AI, specifically the RoBERTa model. RoBERTa, an open-source language model developed by Facebook AI, was fine-tuned to rate personal qualities based on human assessments. Trained research assistants and experienced admissions officers evaluated 3,131 essays, scoring them on a binary scale (0 for ‘No’ and 1 for ‘Yes’) for seven personal qualities: pro-social orientation, leadership, learning, goal pursuit, intrinsic motivation, teamwork, and perseverance.
RoBERTa’s success in this task was reflected in its significant correlations with human ratings. For pro-social orientation and leadership, the correlation was an impressive 0.86 and 0.81, respectively. This demonstrated that the algorithm’s assessments aligned with human evaluators’ assessments, making it a reliable tool for measuring personal qualities.
Eliminating bias and promoting diversity
One of the most remarkable aspects of Luttges’ AI algorithm is its ability to reduce bias and noise in the evaluation process. Unlike humans, the algorithm remains consistent in its judgment, free from the influence of factors like fatigue or time constraints. Moreover, the AI algorithm focuses on specific words and phrases indicative of personal qualities, disregarding other aspects of writing that may reveal socio-economic status and potentially trigger biases.
The AI algorithm’s ability to remain impartial to demographics can contribute to creating more diverse student bodies. The algorithm generates scores unrelated to a student’s background, making it an effective tool for promoting diversity. This stands in contrast to traditional standardized tests like the SAT, which have been criticized for perpetuating biases towards wealthier and predominantly white students.
The utilization of AI in assessing personal qualities in college admissions essays represents a promising step toward fostering diversity on college campuses. The algorithm’s impartiality and consistency offer a fairer and more equitable approach to evaluating applicants, ensuring that admissions decisions are based on personal qualities rather than socio-economic or demographic factors. As higher education institutions seek to build more inclusive communities, AI may be a valuable ally in achieving that goal.
