Overview
Research introduces VisionCoach, an input-adaptive reinforcement learning (RL) framework designed to enhance spatio-temporal grounding in models performing video reasoning tasks. The framework addresses challenges in achieving reliable spatio-temporal grounding, particularly when models are required to locate and track question-relevant evidence across video frames. VisionCoach utilizes visual prompting as a training-time guidance mechanism to amplify pertinent visual information and suppress distractors within challenging inputs, subsequently enabling models to internalize these improvements for grounded reasoning without requiring prompts during inference.
Research Context
Video reasoning necessitates models to accurately identify and follow question-relevant evidence across various frames. While reinforcement learning with verifiable rewards has been shown to improve accuracy in such tasks, it often encounters difficulties in establishing robust spatio-temporal grounding. Current methods to enhance grounding typically involve either scaling up training data, which increases annotation costs, or employing inference-time perception tools, which adds to computational costs.
Approach
VisionCoach operates as an input-adaptive RL framework. During the reinforcement learning training phase, visual prompts are applied selectively to inputs identified as challenging. These prompts function to amplify question-relevant evidence within the video content and simultaneously suppress distracting elements. The model then internalizes the improvements derived from this guided prompting through a process of self-distillation. This internalization allows the model to perform grounded reasoning directly on raw videos during inference without the continued application of visual prompts.
The VisionCoach framework comprises two main components:
- Visual Prompt Selector: This component is responsible for predicting the appropriate type of visual prompt to apply, based on the specific video and question context.
- Spatio-Temporal Reasoner: This component is optimized using reinforcement learning. Its optimization process is guided by the visual prompts and further incorporates object-aware grounding rewards. These rewards specifically enforce object identity consistency and promote multi-region bounding-box overlap to enhance grounding precision.
Findings
Experiments indicate that VisionCoach achieves state-of-the-art performance under comparable settings across a range of benchmarks. These benchmarks include diverse tasks related to video reasoning, video understanding, and temporal grounding, specifically: V-STAR, VideoMME, World-Sense, VideoMMMU, PerceptionTest, and Charades-STA. The framework maintains a single, efficient inference pathway, operating without the need for external tools.
Specific findings highlight that:
- Visual prompting applied during the training phase contributes to improved grounded video reasoning.
- The self-distillation mechanism enables the model to internalize the enhanced grounding ability, eliminating the requirement for visual prompts at inference time.