Exploring the Efficacy of ChatGPT as a Language Learning and Assessing Tool

Exploring the Efficacy of ChatGPT as a Language Learning and Assessing Tool

This study aims to assess how well ChatGPT-3.5 performs on the English reading comprehension section of the College Scholastic Ability Test (CSAT, henceforth), with a focus on its viability as a tool for language learning and evaluation. To do this, tests were conducted where ChatGPT answered questions from the 2022 to 2024 CSAT English exams, and its accuracy was recorded. The research also looks into how ChatGPT’s performance changes depending on the type of question and the version of the AI. The results showed that ChatGPT-3.5 correctly answered about 74.1% of the questions, whereas GPT- 4 had nearly flawless accuracy. The research outlines the capabilities and shortcomings of the large language model (LLM) and discusses areas that require enhancement for educational use. Furthermore, the research examined if these trends hold true for other international English language tests, specifically using reading questions from the TOEFL iBT. Similar performance trends were observed with the TOEFL iBT. The findings indicate that the LLM’s abilities in reasoning and explanation could benefit language education. However, it also emphasizes the necessity for ChatGPT to acquire more specific knowledge and improved reasoning abilities to understand context better.