cs.CVEmpirical
From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models
Jul 7, 2026
This paper proposes ReChannel, a method for dense prediction using a pretrained DiT model, which keeps the encoder but removes the decoder and adapts it with task LoRA. ReChannel maps each token to its corresponding pixel-space patch through a shared linear head.
Proposed a new method for dense prediction using a pretrained DiT model, which involves keeping the encoder but removing the decoder and adapting it with task LoRA.
Before reading this…
Applications
To understand this paper, make sure you know these concepts first:
Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Existing generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets. We argue this inherits more of the generative output interface than dense prediction requires: unlike RGB synthesis, dense prediction asks for pixel-correct, task-native fields on the same image plane, not new RGB content to be rendered. Our key observation is that a pretrained DiT already organizes RGB inputs through a patch-to-token-to-patch lattice on the image plane, so each token indexes a fixed output patch whose channels can carry task-native quantities instead of RGB appearance. We instantiate this as ReChannel: we keep the VAE encoder for the DiT's input distribution but drop the target-side decoder, adapt the frozen DiT with task LoRA, and map each token to its p x p x K_t pixel-space patch through a shared token-local linear head--about 33K parameters, no spatial mixing. Using FLUX-Klein, we evaluate on six dense prediction tasks and over a dozen benchmarks. This minimal interface sets new state-of-the-art on trimap-free matting, KITTI depth, and referring segmentation, and stays competitive on normals, saliency, and pose. In a matched 4B setting it is more accurate and 2.48x faster than an edit-plus-latent-decode counterpart--dense perception can benefit from generative pretraining without inheriting its output interface.
Vision as Unified Multimodal Generation
The paper presents SenseNova-Vision, a unified multimodal model for computer vis…
From Tokens to States: LLMs as a Special Case of World Models and the Continuous Path Beyond
The paper argues that large language models (LLMs) are a special case of world m…
QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents
The paper introduces QVal, a training-free testbed for evaluating dense supervis…
VIDAR: Visual-Inertial Dense Alignment and Reconstruction via a Geometric Foundation Model
This paper introduces VIDAR, a framework for metric dense monocular reconstructi…
Semantic Browsing: Controllable Diversity for Image Generation
This paper introduces a method for controlled diversity in text-to-image models,…
The State-Prediction Separation Hypothesis
The authors propose a Transformer variant that separates state prediction from n…
MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Prediction
This paper presents MotionForesight, a method for predicting future 3D trajector…
Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers
This paper introduces the Bridge, a lightweight interface that enables distillat…