Elastic monitoring framework for cloud infrastructures

This study presents a scalable and elastic distributed system for monitoring cloud infrastructure based on a pure peer-to-peer architecture. Its distributed nature enables deploying long-living queries across the network to monitor a diverse set of entities and metrics, spanning across all layers of a cloud stack that can change rapidly. This allows for aggregating low-level metrics from operating systems, to higher-level application-specific metrics derived from services, databases or application log files. The observed metrics and information can be evaluated and used to reliably trigger policies to automate complex management tasks within a cloud environment. The architecture incorporates a query framework for obtaining high-level information and a policy framework to provide self-management capabilities to monitored cloud infrastructure. The system has been implemented as a proof of concept. Details and statistical results are provided to validate the scalability of the underlying architecture.