Assessing the readability of fiction: A corpus analysis and readability ranking of 200 English fiction texts

Standard readability measures are based on the readability of non-fiction texts. This means that the validity of the measures when applied to fiction texts is questionable. Thus, the scores given to fiction texts using such indices may be invalid when used by English teachers to identify fiction texts of appropriate difficulty for students with various reading ability levels. This paper attempts to address this problem by 1) developing a readability measure specifically designed for fiction texts and 2) applying it to 200 English fiction texts. A corpus, consisting of 100 adults and 100 children s texts, is used for the analysis. In the initial modeling, several standard readability measures are used as variables, and machine learning is used to create a classifier which is able to classify the corpus with an accuracy of 84%. A second classifier is then created using linguistic variables rather than standard readability measures. The latter classifier is able to classify the corpus with an accuracy of 89%, indicating that the standard readability measures are less accurate in classifying fiction texts than linguistic variables. Due to its higher accuracy, the latter classifier is then used to provide a linear complexity or readability rank for each text. The ranking using the linguistic-based classifier provides an more accurate method of determining which texts to choose for students according to their reading levels than the standard readability measures. Importantly, the ranking instantiates a fine-grained increase in complexity. This means that the ranking can be used by an English teacher to select a sequence of texts that represent an increasing challenge to a student without there being a frustratingly discrete rise in difficulty.