通过场景感知文档合成增强关键信息提取中的大型多模态模型 / Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
1️⃣ 一句话总结
本文提出了一种名为SAYRE的文档合成方法,能自动从少量示例文档中学习版面布局和内容规律,生成大量逼真的训练数据,从而有效提升小型多模态模型在关键信息提取任务上的性能,使其在本地设备上也能达到接近云端大模型的精准度。
Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.
通过场景感知文档合成增强关键信息提取中的大型多模态模型 / Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
本文提出了一种名为SAYRE的文档合成方法,能自动从少量示例文档中学习版面布局和内容规律,生成大量逼真的训练数据,从而有效提升小型多模态模型在关键信息提取任务上的性能,使其在本地设备上也能达到接近云端大模型的精准度。
源自 arXiv: 2607.04636