Kajetan Schweighofer
3 indexed papers
Recent (6 mo)
3With code
0Influential cites
0Benchmarked
0Publications per year
326
Top categories
ML×3AI×3
Frequent co-authors
Research Timeline
2026
Efficient Pre-Training of LLMs through Truncated SVD Layers
The paper introduces TSVD, a novel framework that efficiently pre-trains LLMs by enforcing both low rank and strict weight orthonormality, achieving performance comparable to full-parameter models with significantly reduced computational cost.
Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
This paper introduces Anchored Weight Decay (AWD), a regularization technique that effectively prevents prior-task forgetting during LLM fine-tuning with Evolution Strategies (ES), positioning ES as a viable method for continual learning.
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
This paper introduces RREDCoT, a method for approximating optimal reward redistribution in Chain-of-Thought reasoning language models without additional generation.
Highlighted terms show continued research focus across papers