Overview
This study introduces Atomic Physical Transitions (APTs) as a framework for causal video-language understanding. APTs are defined as minimal, temporally localized state changes that link a visible cue to an active physical mechanism and define before/after dynamical regimes. The objective of APTs is to make the hidden process of physical validity explicit, moving beyond clip-level labels that describe what happened to explain why it happened through an ordered causal transition sequence, or APT chain.
Research Context
Traditional video understanding often relies on aggregate event labels (e.g., “bounce”), which, while correct, may not elucidate the underlying causal state changes. These state changes can include specific physical processes such as support loss, contact onset, rebound, and settling. This limitation in existing methods motivated the development of APTs to provide a more granular, physically valid representation of video events.
Approach
To enable Vision-Language Models (VLMs) to learn APTs, a mixed-source dataset was constructed. This dataset incorporated both human annotations and simulator ground truth. The dataset covers 14 types of transitions, specifically related to contact, gravity, friction, and rotation/stability. It comprises 27,303 timed instances observed across 1,246 trials.
The research evaluated existing VLMs on their ability to understand transition-level physics using this dataset. Subsequently, the study investigated direct fine-tuning of these models on APT chains. Following observations from this fine-tuning, a novel parameter-efficient method named APT-Tune was developed. APT-Tune integrates three components:
- Image-pad-aware supervision
- Format-conditional co-training
- Mechanism-conditioned domain-to-type decoding
This recipe was designed to facilitate APT learning in VLMs while maintaining format robustness and physical grounding.
Findings
- Current VLMs exhibited limited understanding of transition-level physics, achieving a zero-shot recall of at most 14%.
- Errors in VLM performance were predominantly due to missed transitions within video events.
- Direct fine-tuning on APT chains improved transition detection by VLMs. However, this approach led to event-level forgetting, suggesting that models learned a specialized answer format rather than a reusable physical representation.
- APT-Tune, utilizing 11 million LoRA parameters on Qwen3-VL-2B, demonstrated substantial improvement in APT recall.
- Beyond improving APT recall, APT-Tune also showed improved event-level video transfer.
- These results indicate that APTs function as a human-aligned causal supervision signal for physical video understanding, rather than merely a new answer format.