Scalable architecture for online prioritisation of cyber threats

Abstract

Detecting advanced attacks is increasingly complex and no single solution can work. Defenders can leverage logs and alarms produced by network and security devices, but big data analytics solutions are necessary to transform huge volumes of raw data into useful information. Existing anomaly detection frameworks either work offline or aim to mark a host as compromised, with high risk of false alarms. We propose a novel online approach that monitors the behaviour of each internal host, detects suspicious activities possibly related to advanced attacks, and correlates these anomaly indicators to produce a list of the most likely compromised hosts. Due to the huge number of devices and traffic logs, we make scalability one of our top priorities. Therefore, most computations are independent of the number of hosts and can be naively parallelised. A large set of experiments demonstrates that our proposal can pave the way to novel forms of detection of advanced malware.