다국면 Rasch 측정의 Facets 프로그램을 활용한 영어 작문 평가의 원어민 채점 검증

An investigation into the native raters' scoring of English writing assessment using the new Facets of many-facet Rasch measurement

The purpose of this paper is to investigate the reliability of English writing test scoring using the computer program of Facets. Based on the many-facet Rasch measurement (MFRM) model, the Facets program has recently been updated and this research uses it in an attempt to analyze the scoring reliability of native raters for an EFL writing test. For the research data the EFL compositions written by 114 English major students were collected and rated by three native raters using a three-category rating scale (Content, Rhetorical Category and Language). At first a brief theoretical basis for the Facets is introduced based on the MFRM model. Then the scores of EFL compositions are analyzed using the Facets and its analysis results are interpreted. Many-facet Rasch analysis revealed that the three native raters are at the same level in terms of rating severity. Although a few cases of unreliable scoring of native raters are found in terms of rater measurement report, it appears that the ratings of EFL compositions by the native raters are generally sound. Finally it is suggested that the Facets program of MFRM can offer many potential advantages for investigating scoring reliability and ensuring construct validation of EFL writing assessment.