Evaluating the Automatic Scoring System for EFL Students’ Writing

The purpose of this study was to evaluate the performance of the automatic scoring system KASS developed by the Korea Institute of Curriculum & Evaluation (KICE). KASS was developed to automatically score sentence‐level compositions of English as foreign language (EFL) students. KASS conducts morphological and syntactical analyses, computes similarity between student writing and correct sentences, and then produces scores based on the scoring rubric employed by human raters (HR). The performance of KASS was evaluated by comparing its scores with those of HR in terms of various agreement indices and correlations. The results indicated that KASS performed well and was a promising automatic scoring system for English sentences constructed by EFL students.