菜单

🤖 系统
📄 Abstract - SAM2S: Segment Anything in Surgical Videos via Semantic Long-term Tracking

Surgical video segmentation is crucial for computer-assisted surgery, enabling precise localization and tracking of instruments and tissues. Interactive Video Object Segmentation (iVOS) models such as Segment Anything Model 2 (SAM2) provide prompt-based flexibility beyond methods with predefined categories, but face challenges in surgical scenarios due to the domain gap and limited long-term tracking. To address these limitations, we construct SA-SV, the largest surgical iVOS benchmark with instance-level spatio-temporal annotations (masklets) spanning eight procedure types (61k frames, 1.6k masklets), enabling comprehensive development and evaluation for long-term tracking and zero-shot generalization. Building on SA-SV, we propose SAM2S, a foundation model enhancing \textbf{SAM2} for \textbf{S}urgical iVOS through: (1) DiveMem, a trainable diverse memory mechanism for robust long-term tracking; (2) temporal semantic learning for instrument understanding; and (3) ambiguity-resilient learning to mitigate annotation inconsistencies across multi-source datasets. Extensive experiments demonstrate that fine-tuning on SA-SV enables substantial performance gains, with SAM2 improving by 12.99 average $\mathcal{J}$\&$\mathcal{F}$ over vanilla SAM2. SAM2S further advances performance to 80.42 average $\mathcal{J}$\&$\mathcal{F}$, surpassing vanilla and fine-tuned SAM2 by 17.10 and 4.11 points respectively, while maintaining 68 FPS real-time inference and strong zero-shot generalization. Code and dataset will be released at this https URL.

顶级标签: medical computer vision model training
详细标签: surgical video segmentation long-term tracking interactive segmentation domain adaptation benchmark dataset 或 搜索:

📄 论文总结

SAM2S:通过语义长期跟踪实现手术视频中的任意分割 / SAM2S: Segment Anything in Surgical Videos via Semantic Long-term Tracking


1️⃣ 一句话总结

这项研究提出了一个名为SAM2S的智能手术视频分割系统,通过增强长期跟踪和语义理解能力,能够实时、精准地分割手术中的器械和组织,显著提升了现有技术的性能。


📄 打开原文 PDF