Back to Paper
cs.LGcs.AIcs.CR
Local ID: 2604.15851v3
AI Summary: gemma4:e4b
DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy
By Erchi Wang, Pengrun Huang, Eli Chien, Om Thakkar, Kamalika Chaudhuri, Yu-Xiang Wang, Ruihan Wu
Revision History Timeline
v14/17/2026
4/17/2026
No submitter comment provided.
v25/15/2026
5/15/2026
No submitter comment provided.
v35/18/2026
5/18/2026
No submitter comment provided.
★ Version indexed in ExplorerComparing v2 vs v3
Green = Added • Red = Removed
Title Comparison
DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy
Authors Comparison
No author changes.
v2 Comment
No comment for this version.
v3 Comment
No comment for this version.
Abstract Word Diff
Differential privacy (DP) has a wide range of applications for protecting data privacy, but designing and verifying DP algorithms requires expert-level reasoning, creating a high barrier for non-expert practitioners. Prior works either rely on specialized verification languages that demand substantial domain expertise or remain semi-automated and require human-in-the-loop guidance. In this work, we investigate whether large language models (LLMs) can automate DP reasoning. We introduce DPrivBench, a benchmark in which each instance asks whether a function or algorithm satisfies a stated DP guarantee under specified assumptions. The benchmark is carefully designed to cover a broad range of DP topics, span diverse difficulty levels, and resist shortcut reasoning through trivial pattern matching. Experiments show that while the strongest models handle textbook mechanisms well, all models struggle with advanced algorithms, revealing substantial gaps in current DP reasoning capabilities. Through further analytic study and failure-mode analysis, we identify several promising directions for improving automated DP reasoning. Our benchmark provides a solid foundation for developing and evaluating such methods, and complements existing benchmarks for mathematical reasoning.