Availability of sentences generated by local large language models as supplementary sentences for textbooks: A paired comparison with original textbook sentences
本地大语言模型生成句作为教材补句的可用性：与教材原句的配对比较

摘要

本研究考察本地大语言模型生成句在教材补句任务中的适用性。从五本英语教材中选取1,969个含情态词的句子，在保留前后文的条件下，由三个模型生成替代句，并比较生成句与教材原句的语言特征和语境适配表现。5,907个最终保留的生成句中，5,864句满足任务要求。相较于教材原句，Mistral与Gemma的句子较短，Aya的句子较长。三个模型的篇章连贯和语义兼容评分与教材原句接近，多数配对评分相同，但不同评价模型对同一句子的判断存在分歧。结果表明，本地模型能够提供具有一定语言质量与语境适配性的补句候选，其适用性取决于模型的表达特点及生成句与前后文的联系。

Abstract

This study examines the applicability of sentences generated by local large language models for the task of textbook supplementary sentence creation. A total of 1,969 modal verb-containing sentences were selected from five English textbooks. With the preceding and following context preserved, three models generated alternative sentences, and the linguistic features and contextual fitness of the generated sentences were compared against the original textbook sentences. Among the 5,907 retained generated sentences, 5,864 met the task requirements. Compared with the original textbook sentences, sentences produced by Mistral and Gemma are shorter, while those from Aya are longer. Scores of discourse coherence and semantic compatibility of the three models are close to those of the original textbook sentences, with most pairs receiving identical scores. However, evaluator models disagreed on judgements for the same sentences. The results show that local models can supply supplementary sentence candidates with acceptable linguistic quality and contextual fitness, and their applicability depends on the expressive characteristics of the models and the links between generated sentences and their surrounding context.