An Interpretation/Use Argument for an ESP Speaking and Writing Test

This paper presents an interpretation/use argument (IUA) based on the argument-based approach to validation (Chapelle, Enright, & Jamieson, 2008; Kane, 1992, 2006, 2013) for TEPS-Speaking & Writing for MOFA (TSW-MOFA), a computer-based English speaking and writing test for Ministry of Foreign Affairs (MOFA) employees in Korea. The construct that the test intends to measure are the English speaking and writing skills needed by MOFA employees to communicate in the workplace. Scores from the test are intended to be used for selection and promotion purposes at the MOFA. The speaking section consists of eight tasks—warming up, answering questions in everyday situations, providing information, TV interview, giving a cultural presentation, making a summary, giving a presentation, and expressing an opinion. The writing section consists of three tasks—essay, translation, and summary. Each task is rated by two independent raters, with the exception of the first speaking task which is unrated. A separate score is given for the two sections of the test. An IUA was constructed for the interpretation and use of scores from the test. The IUA consists of a chain of inferences leading from the test takers’ observed performance on test tasks to the score-based claims and decisions. The five inferences included in the TSW-MOFA IUA are domain definition, evaluation, generalization, explanation, and extrapolation. Each inference is supported by a warrant and one or more assumptions. The IUA also outlines the backing, that is, the evidence required to support each of the assumptions in the IUA. By combining the IUA and backing, a validity argument can be built for the intended score interpretation and use of the test. In the paper, specific evidence is presented as backing wherever possible based on the analysis of available data, including past research reports on the test, test specifications, rater training and calibration materials, and rating data. For backing that is still needed, types of data and data analyses are suggested.