Performance Comparison of LLMs Using PHP and BERT-Based Similarity Models in Maritime English

Effective communication is vital for safe navigation and to promote clarity and consistency in maritime interactions, the International Maritime Organization (IMO) developed the Standard Marine Communication Phrases (SMCP) for use in ship-to-ship and ship-to-shore exchanges. Unlike general English, Maritime English features distinct grammatical structures, vocabulary, and formatting designed for precision and brevity. This study investigates how well commercial Large Language Models (LLMs) can handle Maritime English communication and phrases using PHP Text Similarity algorithms and BERT-based models. Three LLMs (ChatGPT, Google Gemini, and Meta LLaMA 3) were tested using 60 SMCP-based tasks, including sentence formation, terminology interpretation, and fill-in-the-blank questions. Their performance was then compared with that of maritime high school students results. This research is expected to highlight the potential use of general LLMs in maritime contexts.