Yiru Yang
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
The paper proposes a novel layer and point-wise projection mapping combined with LoRA injection to efficiently distill knowledge from a large teacher model to a small student model, significantly improving performance while drastically reducing trainable parameters.
The paper proposes a novel layer and point-wise projection mapping combined with LoRA injection to efficiently distill knowledge from a large teacher model to a small student model, significantly improving performance while drastically reducing trainable parameters.
Papers
Logit Distillation on Manifolds: Mapping by Learning
Yiru Yang, Junling Wang, Nishant Kumar Singh, Luohong Wu +1 more
The paper proposes a novel layer and point-wise projection mapping combined with LoRA injection to efficiently distill knowledge from a large teacher model to a small student model, significantly impr…