Has the style of scientific communication changed due to the growing use of large language models in the writing process? We address this question in the domain of Natural Language Processing by leveraging two data resources we create: a naturalistic corpus of over 37,000 papers from the ACL Anthology (2020-2024); and a synthetic dataset of 3,000 human-written passages and their LLM-generated improvements. We first implement a series of diachronic lexical analyses, showing that both word frequency and usage contexts have changed significantly over time, indicating semantic specialization in some cases and generalization in others. Broadening our perspective, we then model a range of more complex stylistic features and find that LLM-modified texts more frequently contain certain syntactic constructions, more complex and longer words and a lower lexical diversity. Finally, we connect these changes in writing practices to subjective reading experience through a pilot annotation study with 20 domain experts. They overall rate LLM-improved texts as more understandable and exciting, but also express negative qualitative attitudes towards LLMs, highlighting the strongly subjective effect of AI-assisted writing on reading experience.
大语言模型如何改变科学交流？——测量写作实践和阅读体验的变化 / What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience
这篇论文通过分析2020至2024年间自然语言处理领域的大量论文和对比人工写作与AI改进文本，发现大语言模型使学术写作的用词、句法和词汇多样性发生了显著变化，虽然读者认为AI辅助的文本更易懂、更精彩，但他们对这种行为也普遍持负面态度，揭示了AI对科学交流既有客观影响也有主观争议。
源自 arXiv: 2605.19936
