Calvin Yeung
2 indexed papers
Publications per year
Top categories
Frequent co-authors
Research Timeline
The paper introduces Residualized Sparse Autoencoders (ReSAEs) to improve multi-layer interventions in transformers by training each layer on the residual activation, which better preserves cross-layer information relevant to model performance.
The paper introduces residualized temporal Sparse Autoencoders (SAEs) to analyze the full spatiotemporal structure of activations generated during the iterative denoising process of diffusion models, providing a richer understanding than analyzing single timesteps.
Papers
ReSAE: Residualized Sparse Autoencoders for Multi-Layer Transformer Interventions
The paper introduces Residualized Sparse Autoencoders (ReSAEs) to improve multi-layer interventions in transformers by training each layer on the residual activation, which better preserves cross-laye…