Shucheng Li
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
This paper investigates the phenomenon of 'copying' in Distribution Matching Distillation (DMD), finding that high-dimensional distillation causes student models to spontaneously reproduce the teacher's original noise-data pairings due to geometric constraints.
This paper investigates the phenomenon of 'copying' in Distribution Matching Distillation (DMD), finding that high-dimensional distillation causes student models to spontaneously reproduce the teacher's original noise-data pairings due to geometric constraints.
Papers
Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
This paper investigates the phenomenon of 'copying' in Distribution Matching Distillation (DMD), finding that high-dimensional distillation causes student models to spontaneously reproduce the teacher…