This paper proposes a novel video deduplication framework that efficiently reduces online repetition rates while maintaining high precision.
The proposed framework introduces a novel multi-level representation and spatial-temporal matching approach for video deduplication, which improves efficiency and precision compared to existing methods.
Before reading this…
Applications
To understand this paper, make sure you know these concepts first:
The explosive growth of user-generated video content on online platforms is accompanied by the emergence of numerous near-duplicate videos--videos that are identical or highly similar but differ by partial edits. These duplicates degrade user experience and increase storage and bandwidth costs, making large-scale video deduplication a critical task. Existing video deduplication frameworks face a fundamental challenge in retrieving sufficient high-quality candidates under a limited index budget, as well as trade-offs between efficiency and precision. To address these issues, we propose MLT-Dedup, an efficient large-scale online video deduplication framework with Multi-Level representations and spatial-Temporal matching. Our approach employs a Multi-Level Video Encoder (ML-VE) to extract both fine-grained frame-level and sparse clip-level embeddings: sparse embeddings support efficient candidate retrieval, while fine-grained embeddings are loaded for precise pairwise matching. During matching, we introduce DiF-SiM, a Differential Feature-enhanced Similarity Module capable of locating duplicated temporal segments and providing reliable similarity evidence to support policy-driven deduplication decisions. Extensive experiments on a real-world large-scale platform demonstrate that MLT-Dedup reduces online repetition rates by 91% at 90% precision. Furthermore, our sparse retrieval design achieves a 5x increase in indexing capacity, enabling broader candidate coverage in real-world deployment.
Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
EvoPilot is a human-gated method for long-horizon online autoresearch that enabl…
MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos
The authors release a large-scale dataset of raw videos, MultiVENT-Raw, to facil…
WiFi based Multi user Activity Recognition via User Conditioned Spatial Attention
The paper proposes a method for WiFi-based human activity recognition in multi-u…
DyMD: Preserving Interaction Dynamics through Distribution Matching Distillation in Few-Step Video W…
The paper proposes a new framework, DyMD, to improve video generation and embodi…
Online Learning via Learned Latent Bayesian Tracking
This paper proposes AURA, a meta-learning framework for efficient online adaptat…
UEmbed: Unified Sparse and Dense Multimodal Embeddings
The paper introduces UEmbed, a decoder-only multimodal embedding model that prod…
Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning
This paper introduces Surgical WAM, a generative model that learns surgical visu…
MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval
This paper proposes MAGIC, a training-free post-hoc compressor for efficient ret…