Investigating Linguistic Differences in Adult EFL Learners’ Writing Test Performance

Previous research on L2 writing assessment has tended to focus on quantitative analyses of ratings assigned by raters/teachers. Different from previous research, the present study aimed to examine the linguistic features in test-takers’ written responses across different writing task types, focusing on those used for coherence and cohesion in writing. A total of 143 adult EFL learners were presented with two types of writing-compare/contrast and problem/solution-and they were required to write a 300-word essay on one type of their choice. Their written responses (70 compare/contrast and 73 problem/solution) were analyzed using Coh-Metrix to compare the linguistic differences between the two types of writing. Two raters who evaluated both types of writing were further interviewed to examine their perceptions about the discourse organization of the two types. The results indicated that the two types of written responses showed linguistic differences mostly in referential cohesion (e.g., overlap). However, neither of the raters considered referential cohesion when scoring discourse organization; consequently, the use of referential cohesion by the compare/contrast writing group was not reflected in the ratings. The study also suggests the need to consider the time restrictions and task effects on test-takers’ relative difficulty in organizing discourse.