Distractor Analysis to Improve the Quality of Multiple-Choice Item Development

Distractor Analysis to Improve the Quality of Multiple-Choice Item Development

Generating effective distractors is critical to multiple-choice item development because distractors impact both psychometric properties and validity arguments. While the importance of distractors is recognized, studies on the extent distractors function as intended are limited. The purpose of this study is to demonstrate a distractor analysis. To do that, we examine the distractors of multiple-choice items developed to assess U.S. elementary preservice teachers’ metalinguistic knowledge in the four domains of phonology, morphology, syntax, and discourse. We analyze the functioning of the distractors using data from a pilot test of the measure (n=144), employing simple response option selection frequencies and the average scores associated with the selection of each option in the form of Rasch person ability estimates. The results show potential areas for item improvement and demonstrate the value of distractor analysis for making decisions about item revision. Through this analysis, we highlight the significance of distractor analysis in multiple-choice item development, suggesting that teachers with the appropriate content knowledge should be able to conduct such analyses for enhancing language teaching and assessment.