Erik Nielsen
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
The paper proposes SubFit, a novel compression technique that achieves superior LLM compression by replacing non-contiguous, submodule-level components (Attention and FeedForward) with lightweight residual bypasses, significantly improving the perplexity-accuracy trade-off.
The paper proposes SubFit, a novel compression technique that achieves superior LLM compression by replacing non-contiguous, submodule-level components (Attention and FeedForward) with lightweight residual bypasses, significantly improving the perplexity-accuracy trade-off.
Papers
From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression
The paper proposes SubFit, a novel compression technique that achieves superior LLM compression by replacing non-contiguous, submodule-level components (Attention and FeedForward) with lightweight res…