菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-07-06
📄 Abstract - Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.

顶级标签: multi-modal data model training
详细标签: key information extraction document synthesis scene-aware error-driven generation scalable training data 或 搜索:

通过场景感知文档合成增强关键信息提取中的大型多模态模型 / Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis


1️⃣ 一句话总结

本文提出了一种名为SAYRE的文档合成方法,能自动从少量示例文档中学习版面布局和内容规律,生成大量逼真的训练数据,从而有效提升小型多模态模型在关键信息提取任务上的性能,使其在本地设备上也能达到接近云端大模型的精准度。

源自 arXiv: 2607.04636