Test Fairness

The purpose of this study is to investigate whether an EFL test was associated with DIF across gender and to locate the source of DIF in question types and passage content. Findings showed that even though females obtained significantly higher scores than males in 48 items out of 50 in the group mean comparison, 15 items with six items in favor of males and nine items in favor of females showed DIF after matching ability levels. In the source of DIF, males had higher probabilities of getting items about inferential questions correct in listening comprehension, whereas females had higher probabilities of getting items about literal understanding and applications (indirect assessment of speaking ability) correct. Interestingly, the reverse held true in reading comprehension. That is, questions regarding literal understanding and application (indirect assessment of writing ability) differed in their functioning in favor of males, whereas questions regarding inferences, main ideas, and grammar differed in their functioning in favor of females. In passage content, more female-favoring contents than male-favoring contents in quality as well as in quantity were found, probably leading to the poor performance by males in the reading test. These findings imply that item developers take every effort to take a rational choice between question types and passage content in order to develop a fair test across subgroups such as gender.