A Comparison and Analysis of CSAT Reading Texts Before and After the Use of Criterion-Referenced Assessment (2017 vs. 2018): A Focus on Lexical Complexity

The Ministry of Education in Korea (2014) decided to adopt a criterion-referenced assessment approach in the English language section of the 2018 College Scholastic Adaptability Test (CSAT). Accordingly, it became particularly important to ensure a consistent level of difficulty in the English language test across years. Unlike the types of questions, different reading and listening texts are presented each year. Due to the advancement of technology, however, it is comparatively easy to predict their difficulty levels in advance. In this study as part of a larger research project (see Chang (2018)), three categories of lexical factors (lexical density, lexical sophistication, and lexical variation) were compared between the CSAT English reading passages before and after the use of criterion-referenced assessment (2017 and 2018). Fifty reading passages were extracted from the 2017 and 2018 CSAT and were compiled into different corpora, respectively. In total, twenty-five indices from a lexical complexity analyzer (Ai & Lu, 2010; Lu, 2012) were used (one measure for lexical density, five measures for lexical sophistication, and nineteen measures for lexical variation). Independent samples t-tests were conducted, and their results show that there were no statistically significant differences between 2017 and 2018 CSAT reading texts in lexical density, lexical sophistication, and lexical variation. The implications of these results will be discussed.