생성형 인공지능 영어 글쓰기 평가 도구 간 비교 연구: ChatGPT, Perplexity, Gemini, Grammarly를 중심으로

A Comparative Study of Generative AI-Based English Writing Evaluation Tools: Focusing on ChatGPT, Perplexity, Gemini, and Grammarly

This study compared four generative AI-based English writing evaluation tools—ChatGPT, Perplexity, Gemini, and Grammarly—by analyzing their assessments of Korean university students’ essays according to Grammarly’s five criteria: accuracy, clarity, conci- seness, tone, and formality. Forty first-year pre-medical students enrolled in a general English writing course participated in the study. Each essay was evaluated by all tools using the same prompt to obtain criterion scores (1-20) and overall scores (100). Descriptive statistics and repeated-measures ANOVA were used to test mean differences across tools, while Friedman tests and Kendall’s coefficient of concordance examined rank consistency. According to the results, ChatGPT demonstrated the highest overall mean being followed by Perplexity and Grammarly, whereas Gemini exhibited the lowest mean score. Significant differences were found in accuracy, clarity, conciseness, and formality (p<.001), but not in tone (p=.132). Nonparametric tests revealed no significant differences or agreement in overall rank, suggesting convergence in holistic evaluation tendencies. Findings indicate the pedagogical value of integrating multiple AI tools with human assessment, selecting tools aligned with rubric priorities, and guiding students to interpret and apply AI feedback effectively in formative writing cycles.