The paper proposes a novel, locally deployable agentic workflow using large language models (LLMs) to accurately and privately detect various types of personally identifiable information (PII) within unstructured crash narratives.
Crash narratives in crash reports provide crucial contextual information for traffic safety analysis. Yet, their broader use is hindered by the presence of personally identifiable information (PII), including names, home addresses, and license plate numbers. Because PII appears sparsely and inconsistently in crash narratives, manual detection is not scalable, and existing rule-based approaches often fail to capture context-dependent PII. This study develops and evaluates a locally deployable, agentic workflow for PII detection in crash narratives by leveraging large language models (LLMs). The workflow contains a Hybrid Extractor and a Verifier. The Hybrid Extractor routes structured PII (e.g., phone numbers and email addresses) to a rule-based model (i.e., Presidio) and context-dependent PII (e.g., names, home addresses, and alphanumeric identifiers) to a domain-adapted, fine-tuned LLM. To address ambiguity in challenging categories, the workflow incorporates ensemble LLM extraction and an agentic verification step that filters false detections through evidence-based reasoning. Evaluated on a real-world crash dataset, the agentic workflow achieves strong performance with a precision of 0.82, a recall of 0.94, an F1 of 0.87, and an accuracy of 0.96, outperforming multiple baseline methods. Moreover, the ablation results suggest that ensemble LLM extraction and Verifier offer improved detection for home addresses and alphanumeric identifiers. The workflow runs locally, supporting privacy-sensitive operational settings where external APIs are restricted. This work offers a practical and robust path for scalable, privacy-preserving crash data processing, enabling broader research and safety interventions while safeguarding individual privacy.
Towards Context-Aware Image Anonymization with Multi-Agent Reasoning
The paper introduces CAIAMAR, a multi-agent reasoning framework that achieves co…
WebPII: Benchmarking Visual PII Detection for Computer-Use Agents
The paper introduces WebPII, a novel, large-scale synthetic benchmark for detect…
A Hardware-Anchored Privacy Middleware for PII Sharing Across Heterogeneous Embedded Consumer Device…
The paper proposes the User Data Sharing System (UDSS), a hardware-anchored midd…
Hardening x402: PII-Safe Agentic Payments via Pre-Execution Metadata Filtering
The paper introduces presidio-hardened-x402, an open-source middleware that inte…
Protecting User Prompts Via Character-Level Differential Privacy
The paper proposes a character-level differential privacy mechanism to sanitize…
SEAL-Tag: Self-Tag Evidence Aggregation with Probabilistic Circuits for PII-Safe Retrieval-Augmented…
SEAL-Tag is a privacy-preserving runtime environment that mitigates PII leakage…
Structured Visual Narratives Undermine Safety Alignment in Multimodal Large Language Models
This paper introduces ComicJailbreak, a new benchmark demonstrating that structu…
LLM-Redactor: An Empirical Evaluation of Eight Techniques for Privacy-Preserving LLM Requests
The paper systematically evaluates eight privacy-preserving techniques for LLM r…