Online Safety Monitoring for LLMs
This paper proposes a simple real-time monitor for LLMs that turns an external verifier signal into an alarm decision by thresholding, showing competitiveness with advanced monitors in mathematical reasoning and red teaming datasets.
Proposing a simple and effective method for real-time monitoring of language models' safety.
Before reading this…
Applications
- →Safety monitoring for language models
To understand this paper, make sure you know these concepts first:
- Understanding of language models and safety monitoringfind papers →
Abstract
More Like ThisDespite alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an external model into an alarm decision by thresholding, with the threshold calibrated via risk control. In experiments on mathematical reasoning and red teaming datasets, we show that this simple design is competitive with more advanced monitors based on sequential hypothesis testing.