A Comparison of Text Difficulty in 2015-2020 CSAT English Reading Passages between Testing Types and among Question Types

The current study investigates the degree of text difficulty in CSAT English reading passages between 2015-2017 norm- and 2018-2020 criterion-referenced testing, and among question types including understanding general ideas, details, and logical relation. In order to compare the text difficulty, the corpus was constructed from 150 reading passages in the 2015-2020 CSAT. Coh-Metrix and professional Lexile analyzer analyzed a basic count, readability levels, and text easability levels. Concerning the comparison between two testing types, the sentence length is significantly different in understanding details, and the Coh-Metrix L2 readability and Lexile measure indicate significant differences only in understanding logical relation. Considering the comparison among the three question types, sentence length is significantly different among the three question types in both testing types. The criterion-referenced testing shows a clear difference among the three question types in all indices of readability levels while the norm-referenced testing does not show any obvious distinction among the question types in any index. The norm- and criterion-referenced testing types show different trends of text easability across the three question types. Overall, the differences between the two testing types do not seem clear without classifying reading passages into the three different question types. The pedagogical implications are included.