VICX: Generalizable Robot Manipulation via Video Generation and In-Context Operator Network
VICX proposes a decoupled closed-loop manipulation framework for generalizable robot manipulation.
VICX introduces a novel decoupled closed-loop manipulation framework that combines video generation and in-context execution for generalizable robot manipulation.
Before reading this…
Applications
- →robot learning
- →robot manipulation
To understand this paper, make sure you know these concepts first:
- roboticsfind papers →
- computer visionfind papers →
- machine learningfind papers →
Abstract
More Like ThisGeneralizable robot manipulation requires not only task-level reasoning over unseen scenes, but also reliable grounding of visual plans into embodiment-specific execution. To bridge this gap, we propose VICX (Video generation and In-Context eXecution), a decoupled closed-loop manipulation framework. In VICX, a frozen video generation model produces vision-language-conditioned high-level visual plans, while a Video-to-Trajectory In-Context Operator Network (V2T-ICON) serves as the task-agnostic interface that grounds these plans into executable robot-state trajectories. To improve execution generalization, V2T-ICON operates on segmentation-extracted arm-only frame observations and uses retrieved image-state pairs as in-context prompts, allowing a robust and generalizable visual-to-state mapping at inference time without parameter updates. Experiments on Meta-World show that VICX supports cross-task generalization, closed-loop self-correction, and cross-embodiment transfer, demonstrating dual generalization across both task semantics and robot execution. The project webpage can be found here: https://scaling-group.github.io/vicx/.