BERT takes the test: automated essay scoring in the age of AI

This study investigates the effectiveness and fairness of a fine-tuned BERT model for automated essay scoring in higher education. This research demonstrates that fine-tuning an open-source language model on a relatively small and balanced dataset, yielding highly accurate and unbiased results. The model was trained and evaluated using real exam data from a university-level International Management course, with a focus on replicating human grading behavior and testing for potential bias. Regression analyses revealed a strong correlation between predicted and human-assigned grades, and no statistically significant bias related to students’ language background or special educational needs. These findings suggest that fine-tuned models can serve as reliable, equitable, and scalable tools for automating essay assessment in higher education, offering a promising complementary alternative to traditional grading methods. This study adds novelty by moving beyond synthetic or secondary-school contexts and applying fine-tuned automated grading to real higher education exam data. It also contributes by testing for potential bias related to student characteristics, an underexplored dimension in such research settings.