菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-04-29
📄 Abstract - What Kind of Language is Easy to Language-Model Under Curriculum Learning?

Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impossible languages to very common combinations of features (e.g., subject-object-verb word order). One central question is under what conditions such typological tendencies can be predicted, and specifically whether the learning bias of language models (LMs) is sufficient to reproduce such patterns. In this study, we add one dimensionality to such analysis -- the learning scenario for LMs -- to explore its interaction with the inductive bias of LMs. Specifically, as a first study, we examine the effect of curriculum learning (CL), as a developmentally motivated learning scenario, i.e., starting with simpler sentences rather than randomly-ordered input. We expand existing LM-based exploration (El-Naggar et al., 2025a,b) with a simple CL variant and find that CL substantially impacts the apparent inductive bias of LMs.

顶级标签: llm natural language processing model training
详细标签: curriculum learning language typology inductive bias word order 或 搜索:

在课程学习下,什么样的语言对语言模型来说是容易的? / What Kind of Language is Easy to Language-Model Under Curriculum Learning?


1️⃣ 一句话总结

这项研究通过让语言模型先学习简单句子再接触复杂句子(课程学习),发现这种类似人类的学习顺序会显著改变模型对不同语言类型的偏好,从而部分解释了为什么现实世界中某些语言特征组合更加常见。

源自 arXiv: 2604.26844