Overview
The USV (User-generated Short-form Video) dataset has been developed to facilitate research into user-generated short-form videos. This dataset is specifically designed for high-level semantic video understanding. It is comprised of approximately 224,000 videos that were collected from user-generated content (UGC) platforms. The collection process involved using label queries and did not incorporate additional manual verification or trimming of the videos.
The research establishes two distinct tasks within the context of this dataset: topic recognition and video-text retrieval. To address these tasks, two unified and effective baseline methods are proposed: Multi-Modality Fusion Network (MMF-Net) for topic recognition and Video-Text Contrastive Learning (VTCL) for video-text retrieval. The study also includes comprehensive benchmarks to support future research efforts in this domain.
Research Context
Recent years have seen the publication of several large-scale video datasets, which have contributed to advancements in video understanding. However, the study notes that user-generated short-form videos, a newly emerging category, have seldom been the subject of dedicated research. While video understanding has shown improvements, many existing works tend to focus on instance-level recognition. The researchers contend that this focus is insufficient for effectively learning the representation of high-level semantic information present in videos.
Approach
The core of the approach involves the creation and deployment of the USV dataset. This dataset was constructed by collecting videos from UGC platforms using label queries. The collection process did not include manual verification or trimming, indicating an automated or semi-automated acquisition strategy.
Following the dataset's establishment, two specific tasks were defined to explore high-level semantic video understanding: topic recognition and video-text retrieval. For each of these tasks, a dedicated baseline method was developed and proposed:
- Topic Recognition: The Multi-Modality Fusion Network (MMF-Net) was introduced as a baseline method for this task.
- Video-Text Retrieval: The Video-Text Contrastive Learning (VTCL) method was proposed as a baseline for this specific task.
The research subsequently carried out comprehensive benchmarks utilizing these proposed methods to facilitate and guide future investigations into user-generated short-form videos and high-level semantic understanding.
Findings
- The USV dataset, containing approximately 224,000 videos collected from UGC platforms via label queries without manual verification or trimming, has been formally presented.
- Two distinct tasks critical for high-level semantic understanding of user-generated short-form videos, topic recognition and video-text retrieval, have been established on the USV dataset.
- MMF-Net has been proposed as a unified and effective baseline method specifically for the topic recognition task.
- VTCL has been proposed as a unified and effective baseline method specifically for the video-text retrieval task.
- Comprehensive benchmarks have been conducted using these proposed methods, with the aim of facilitating future research.