Roberto Dailey
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
This paper introduces Anchored Weight Decay (AWD), a regularization technique that effectively prevents prior-task forgetting during LLM fine-tuning with Evolution Strategies (ES), positioning ES as a viable method for continual learning.
This paper introduces Anchored Weight Decay (AWD), a regularization technique that effectively prevents prior-task forgetting during LLM fine-tuning with Evolution Strategies (ES), positioning ES as a viable method for continual learning.
Papers
Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
This paper introduces Anchored Weight Decay (AWD), a regularization technique that effectively prevents prior-task forgetting during LLM fine-tuning with Evolution Strategies (ES), positioning ES as a…