📄 论文总结
MPJudge:面向音乐诱导绘画的感知评估 / MPJudge: Towards Perceptual Assessment of Music-Induced Paintings
1️⃣ 一句话总结
本文提出了一种评估音乐与绘画感知一致性的新方法MPJudge,通过构建首个大规模专家标注数据集和引入偏好优化训练,有效解决了现有方法依赖情绪识别而忽略更广泛感知线索的问题。
Music induced painting is a unique artistic practice, where visual artworks are created under the influence of music. Evaluating whether a painting faithfully reflects the music that inspired it poses a challenging perceptual assessment task. Existing methods primarily rely on emotion recognition models to assess the similarity between music and painting, but such models introduce considerable noise and overlook broader perceptual cues beyond emotion. To address these limitations, we propose a novel framework for music induced painting assessment that directly models perceptual coherence between music and visual art. We introduce MPD, the first large scale dataset of music painting pairs annotated by domain experts based on perceptual coherence. To better handle ambiguous cases, we further collect pairwise preference annotations. Building on this dataset, we present MPJudge, a model that integrates music features into a visual encoder via a modulation based fusion mechanism. To effectively learn from ambiguous cases, we adopt Direct Preference Optimization for training. Extensive experiments demonstrate that our method outperforms existing approaches. Qualitative results further show that our model more accurately identifies music relevant regions in paintings.
MPJudge:面向音乐诱导绘画的感知评估 / MPJudge: Towards Perceptual Assessment of Music-Induced Paintings
本文提出了一种评估音乐与绘画感知一致性的新方法MPJudge,通过构建首个大规模专家标注数据集和引入偏好优化训练,有效解决了现有方法依赖情绪识别而忽略更广泛感知线索的问题。