Analyzing Problem-Solving Behaviors via Action Sequence Clustering in Computer-Based Assessment

Paper-based assessments often rely on total scores to evaluate performance, which are insufficient for capturing the complexity of problem-solving processes. People adopt diverse cognitive strategies when approaching a problem. While some may arrive at the correct solution through intended methods, others may do so by chance, indicating that response outcomes alone cannot fully capture people's abilities. This study analyzed action sequences derived from process data to characterize problem-solving approaches, focusing on the “CD tally” item from the Problem-Solving in Technology-Rich Environments module of the PIAAC 2012. This item required participants to meet specific criteria within a spreadsheet environment and submit their responses via a combo box in a web interface. The extraction of n-grams, a technique commonly employed in natural language processing, was used to analyze the participants' action sequences. Subsequently, we use a Kmedoids cluster analysis to identify four distinct clusters of similar action sequences. Each cluster was examined to reveal unique characteristics, such as the number of actions and response times, thereby providing insights into the varied problem-solving strategies among participants.