Count Me In: Viable Distributed Summary Statistics for Securing High-Speed Networks

TitleCount Me In: Viable Distributed Summary Statistics for Securing High-Speed Networks
Publication TypeConference Paper
Year of Publication2014
AuthorsAmann, J., Hall S., & Sommer R.
Other Numbers3675

Summary statistics represent a key primitive for profiling and protectingoperational networks. Many network operators routinely measure properties such asthroughput, traffic mix, and heavy hitters. Likewise,security monitoring often deploys statistical anomaly detectors that trigger,e.g., when a source scans the local IP address range, or exceeds a thresholdof failed login attempts. Traditionally, a diverseset of tools is used for such computations, each typicallyhard-coding either the features it operates on or the specific calculationsit performs, or both.In this work we present a novel framework for calculating a wide array ofsummary statistics in real-time, independent of the underlying data, andpotentially aggregated from independent monitoring points. We focus onproviding a transparent, extensible, easy-to-use interface and implementour design on top of an open-source network monitoring system.We demonstrate a set of example applications forprofiling and statistical anomaly detection that would traditionally requiresignificant effort and different tools to compute. We have released our implementationunder BSD license and report experiences from real-world deployments in large-scalenetwork environments.


This work was partially supported by funding provided to ICSI through National Science Foundation grants OCI : 1032889 (“Enhancing Bro for Operational Network Security Monitoring in Scientific Environments”) and ACI : 1348077 ("A Bro Center of Expertise for the NSF Community"); through the U.S. Army Research Laboratory and the U.S. Army Research Office under MURI grant No. W911NF-09-1-0553; and by the Deutscher Akademischer Austausch Dienst (DAAD) through a postdoctoral fellowship. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors or originators and do not necessarily reflect the views of the National Science Foundation, the U.S Army Research Office, the U.S. Army Research Laboratory, or the DAAD.

Bibliographic Notes

Proceedings of the 17th International Symposium on Attacks, Intrusions, and Detections (RAID 2014), Gothenburg, Sweden

Abbreviated Authors

J. Amann, S. Hall, and R. Sommer

ICSI Research Group

Networking and Security

ICSI Publication Type

Article in conference proceedings