Performance of ChatGPT 3.5 on CSAT: Its Potential as a Language Learning and Assessment Tool

Performance of ChatGPT 3.5 on CSAT: Its Potential as a Language Learning and Assessment Tool

The purpose of this paper is to evaluate ChatGPT 3.5’s performance on the English reading comprehension section of the Korean College Scholastic Ability Test (CSAT). More specifically, this study analyzes its potential as a language learning and assessment tool based on its performance on the high-stakes national test. This study also identifies the strengths and limitations of the large language model (LLM) and explores aspects that need improvement to use for learning. The results suggest that the LLM’s strengths in reasoning and explanation can be useful for language learning. The results also highlight the need for ChatGPT to gain extensive domain-specific knowledge and advanced reasoning skills in order to enhance its contextual understanding. Moreover, the results indicate that human judgment from educators is necessary to ensure the accuracy and reliability of ChatGPT’s outputs, particularly for a high-stakes exam like the CSAT. Combining AI and human judgment can result in more effective and trustworthy results.