Dango:用于研究第二语言习得的严格仅母语大语言模型 / Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition
1️⃣ 一句话总结
本文介绍了一个名为Dango的1.8B参数大语言模型,通过精心过滤训练数据确保模型仅接触母语(日语),再专门用第二语言(英语)学习课程进行微调,从而模拟人类第二语言习得过程,并验证了该方法能生成更接近人类的学习者语言模式。
We introduce Dango, a 1.8B-parameter large language model designed for controlled studies of L1-to-L2 (Japanese-to-English) transfer in second language acquisition (SLA). While previous studies have explored SLA in language models, they have predominantly relied on smaller or non-decoder models, limiting their ability to generate open-ended text and reducing their suitability as practical L2 simulators. We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition. To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure. We then fine-tune the model on LLM-generated L2-learning lessons to simulate the L2 acquisition process. Our evaluations confirm that Dango develops human-like L2 production patterns, outperforming both unfiltered and standard multilingual baselines. We release the model, data, and code to facilitate reproducible computational SLA research and learner-facing applications.
Dango:用于研究第二语言习得的严格仅母语大语言模型 / Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition
本文介绍了一个名为Dango的1.8B参数大语言模型,通过精心过滤训练数据确保模型仅接触母语(日语),再专门用第二语言(英语)学习课程进行微调,从而模拟人类第二语言习得过程,并验证了该方法能生成更接近人类的学习者语言模式。
源自 arXiv: 2606.19170