Differentially Private Log Mining with Utility-Preserving Aggregation for Rare-Event Pattern Discovery

Abstract

Operational software systems continuously emit logs that encode execution traces, security signals, and user-facing failures. These logs are routinely mined to discover patterns that support debugging, incident response, and reliability engineering, yet they also contain sensitive information whose disclosure can enable re-identification, behavioral inference, and leakage of proprietary internals. Differential privacy offers principled protection for aggregate analytics, but rare-event pattern discovery remains difficult because privacy noise and clipping can erase precisely the low-support structures that matter for outages and attacks. This paper develops a log-mining framework that couples differentially private measurement with utility-preserving aggregation tailored to rare events. The approach models logs as structured, multi-resolution summaries that combine count sketches, compressed token-sequence representations, and low-rank embeddings to retain signal at small scales while controlling sensitivity. Utility is formalized through downstream pattern-recovery objectives, and privacy is enforced through calibrated mechanisms with composition-aware accounting. A constrained optimization view links privacy budgets, latency, and compute to the fidelity of recovered rare patterns. The design supports both streaming and batch pipelines, distributed execution, and reproducible evaluation. The resulting system provides practical rare-event discovery under rigorous privacy constraints by aligning mechanism design with the geometry of log features and the combinatorics of pattern search, rather than treating privacy as an after-the-fact perturbation of fixed reports.