LLM-CoT 기반 수능 영어 고난도 문항 생성 연구: 함축 의미 추론 문항 중심으로

A study on generating high-difficulty CSAT English items based on LLMdriven Chain-of-Thought (CoT): Focusing on implicature inference tasks

This study applies Chain-of-Thought (CoT) prompting to large language models (LLMs) to generate high-inference English implicature items for the College Scholastic Ability Test (CSAT) and evaluate their linguistic alignment and validity. Using GPT-4o for prototyping and GPT-5 for final generation, we produced 45 items under three prompt conditions (Basic, Reasoning, Self-review) and compared them with 15 official KICE CSAT items. Corpus-based indices of lexical and syntactic complexity, including type–token ratio, lexical density, and selected syntactic measures, were analyzed with one-way ANOVAs and post-hoc tests. LLM-generated items showed higher lexical complexity and distinct syntactic profiles. Basic prompts most closely resembled CSAT items, whereas Reasoning and Self-review yielded items with greater inferential demands but heavier reliance on local cues. Robustness checks indicated stable lexical patterns, though several advanced syntactic metrics were sensitive to sampling. Overall, CoT prompting functioned not only as a generation aid but as a principled framework for item design, suggesting that practical use should combine lexical difficulty control, strategic cue placement, and a humanin-the-loop plus self-review CoT.