Qian Lou

SecureRouter is an encrypted routing and inference framework that accelerates secure transformer inference by adaptively selecting the optimal model size based on the encrypted input, achieving a 1.95x latency reduction.

The paper proposes PPRoute, a privacy-preserving LLM routing framework that significantly speeds up secure model selection while maintaining high performance comparable to non-private methods.

The paper proposes a novel space switching method to efficiently unify arithmetic and comparison operations within Fully Homomorphic Encryption (FHE) schemes, achieving significant performance improvements over existing techniques.