Towards More General Control of Diffusion Models Using Jeffrey Guidance
The authors propose a framework called Jeffrey guidance to extend diffusion-model control to target distributions beyond simple cases.
The proposed framework, Jeffrey guidance, is novel in its ability to target distributions beyond simple cases, extending the control of diffusion models.
Before reading this…
Applications
- →image generation
- →fairness
To understand this paper, make sure you know these concepts first:
- diffusion modelsfind papers →
- conditional probabilityfind papers →
Abstract
More Like ThisA key strength of diffusion models lies in their flexibility, since their outputs can be controlled at sampling time through guidance. However, beyond simple cases such as conditional sampling, the target distribution is often left implicit, defined only through a sampling rule or a heuristic energy function. To address this, we propose Jeffrey guidance, a principled framework that extends diffusion-model control to applications beyond what standard guidance can express. It leverages Jeffrey's rule of conditioning to update marginal distributions towards a prescribed target, preserving the conditional structure and minimally perturbing the joint distribution. We first demonstrate Jeffrey guidance by targeting a prescribed embedding distribution. With Inception embeddings as the target, this leads to substantial reductions in FID on both CIFAR-10 and FFHQ. We further apply Jeffrey guidance to fairness on CelebA-HQ, updating an unconditional diffusion model to enforce independence between attributes.