Callum McDougall

The paper demonstrates that sparse autoencoders can successfully extract a large set of interpretable, causally influential features from the production-scale Claude 3 Sonnet language model.

This paper investigates the transparency of DiffusionGemma, a model with a larger fraction of computation in a continuous latent space, and shows that it can be made more transparent by mapping information through an interpretable token bottleneck.