Consistent Video-to-Video Translation via Explicit Correspondences
The paper introduces vid2vid-long, a correspondence-based approach to improve long-range temporal consistency in video-to-video translation.
The paper introduces a novel correspondence-based approach to video-to-video translation that improves long-range temporal consistency.
Before reading this…
Applications
- →Interactive video-to-video applications
To understand this paper, make sure you know these concepts first:
- Video-to-video translationfind papers →
- Attention mechanismsfind papers →
Abstract
More Like ThisInteractive video-to-video applications require real-time generation while maintaining long-range temporal consistency. However, recent methods achieve speed by restricting attention to a fixed-length causal window. This limits long-range consistency whenever relevant history falls outside the context window. To address this limitation, we introduce vid2vid-long, a correspondence-based approach grounded in a simple observation. Video-to-video translation provides an explicit cross-frame reference: the regions that match in the input video should match in the output video too. Concretely, we compute patch-level correspondences in the input, guiding retrieval of previously generated outputs in the corresponding regions, and condition the model on these patches through correspondence cross-attention. Our experiments demonstrate that vid2vid-long consistently improves long-range temporal consistency across recent video models while preserving generation quality and real-time performance.