Rethinking the Assessment of Oral Communicative Competence in Higher Education: Integrating Closed-Ended and Open-Ended Questionnaires with Generative AI Tools

The assessment of Oral Communicative Competence (OCC) in university contexts is essential, both for its formative value and its close link to employability. However, its assessment remains infrequent and methodologically challenging. This study explores the self-perceived OCC (SOCC) through two different instruments: a closed-ended questionnaire (Likert-type items) and a set of open-ended questions, both developed around the same theoretical dimensions. A total of 112 university students participated. Open-ended responses (OeR) were analyzed using a GPT-4 model refined using prompt engineering techniques to ensure consistency and subsequently validated by expert raters. To examine the degree of correspondence between the two instruments, random forest models were used to predict the AI-based categorizations of OeR from the scores on the Closed-ended Responses (CeR). The results revealed an overall low accuracy, although three items showed accuracy above 70%, particularly those focused on observable and functional behaviors. These findings suggest that open-ended instruments capture dimensions of competence not typically reflected in closed-ended tools, offering valuable insights for the development of richer, more sensitive, and contextually appropriate assessment strategies in higher education.