Assessing Source Text Difficulty in Interpreter Education : Expert Assessment and the Measurement of Textual Properties

Since STs in interpreter education are commonly selected by teachers, instructors must also determine the right level of the difficulty of STs for classwork and exams in order to ensure the appropriateness of the materials. However, interpreting teachers tend to disagree on the optimal level of difficulty of a given ST, resulting in less reliable assessment results. This study, therefore, addressed two questions related to the subjective determination of ST difficulty by interpreting teachers: 1) how consistent are their assessments with one another?; and 2) what are the textual properties related to their subjective judgments of ST difficulty? 15 interpreting teachers from postgraduate English-Korean interpreting programs rated the difficulty of five English STs and the scores they assigned were compared. Very low inter-rater consistency is found in the subjective judgment of ST difficulty. On the other hand, the participants difficulty assessment of the STs, on the basis of the mean values of their scores, showed significant correlations with the average length of T-unit, Flesch Reading Ease Score and type-to-token ratio of the STs. Thus, subjective evaluating of vocabulary- and sentence-related difficulty should be replaced by these quantifiable indicators.