菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-03-30
📄 Abstract - GEMS: Agent-Native Multimodal Generation with Memory and Skills

Recent multimodal generation models have achieved remarkable progress on general-purpose generation tasks, yet continue to struggle with complex instructions and specialized downstream tasks. Inspired by the success of advanced agent frameworks such as Claude Code, we propose \textbf{GEMS} (Agent-Native Multimodal \textbf{GE}neration with \textbf{M}emory and \textbf{S}kills), a framework that pushes beyond the inherent limitations of foundational models on both general and downstream tasks. GEMS is built upon three core components. Agent Loop introduces a structured multi-agent framework that iteratively improves generation quality through closed-loop optimization. Agent Memory provides a persistent, trajectory-level memory that hierarchically stores both factual states and compressed experiential summaries, enabling a global view of the optimization process while reducing redundancy. Agent Skill offers an extensible collection of domain-specific expertise with on-demand loading, allowing the system to effectively handle diverse downstream applications. Across five mainstream tasks and four downstream tasks, evaluated on multiple generative backends, GEMS consistently achieves significant performance gains. Most notably, it enables the lightweight 6B model Z-Image-Turbo to surpass the state-of-the-art Nano Banana 2 on GenEval2, demonstrating the effectiveness of agent harness in extending model capabilities beyond their original limits.

顶级标签: agents multi-modal model evaluation
详细标签: agent framework multimodal generation memory systems skill learning benchmark evaluation 或 搜索:

GEMS:具备记忆与技能的、以智能体为核心的多模态生成框架 / GEMS: Agent-Native Multimodal Generation with Memory and Skills


1️⃣ 一句话总结

这篇论文提出了一个名为GEMS的智能体框架,它通过引入多智能体协作循环、长期记忆存储和可扩展的专业技能库,显著提升了多模态生成模型在复杂指令和特定任务上的表现,甚至能让轻量级模型超越更强大的模型。

源自 arXiv: 2603.28088