일반화가능도 이론을 적용한 한국어 말하기 성취도 평가의 신뢰도와 오차요인 분석

Analysis of Error Sources and Estimation of Reliability in a Korean Speaking Achievement Test by Applying Generalizability Theory

The purpose of the study is: 1) to determine error sources and the effects of each error source in a Korean speaking achievement test; 2) to estimate the generalizability coefficient; and 3) to suggest optimal measuring conditions for Korean speaking achievement tests based on the generalizability theory. For the design of the G study, a three-facet, partially-nested model (p x (f:t) x r) was used with each of three facets setting tasks, raters, and assessment criteria as random effects. The G study design was used in an identical manner as the D study. As a result of the G study, error sources in the Korean speaking achievement test indicate that error sources for tasks have a more significant impact than error sources related to raters or assessment criteria associated with the score of the speaking test. As for the generalizability coefficients in the D study, increasing the number of tasks had a more significant impact than increasing the number of raters or assessment criteria in the Korean speaking achievement test. Therefore, it seems more efficient to increase the number of tasks rather than the number of raters or assessment criteria in order to create the optimal conditions for a Korean speaking achievement test.