The Latent Trait Modeling of Passage-Based Reading Comprehension Test: Testlet-Based MIRT Approach

Numerous IRT studies have regarded reading comprehension tests as a typical example of testlet-based tests that are organized in a unit referred to as a testlet, a group of items sharing a common stimulus. Testlet-based tests, however, can cause item responses from within the same testlet to be more correlated than across testlets. And the residual correlations create the testlet effect, which is specifically associated with a testlet itself (e.g., prior knowledge about reading passages). This study aims to 1) demonstrate the existence of such effect resulting into the multidimensionality of a reading comprehension test administered to 830 Korean high school students and 2) specify the model which can properly address the testlet effect. For this purpose, testlet-based multidimensional IRT models – bifactor and Testlet Response Theory models – are compared in reference to a unidimensional model. Results confirm the multidimensionality of the test and find the bifactor model to be the preferred one in terms of the discriminating power of the general trait to be measured in the test and the capacity to accommodate the testlet effect that is variable within and between passages.