Technical Report on Cloud Metrics Published

Tuesday, March 8, 2016 05:00

The SPEC RG Cloud Working Group has published a Technical Report on cloud metrics.

In the past decade, cloud computing has emerged from a pursuit for a service-driven information and communication technology (ICT), into a signicant fraction of the ICT market. Responding to the growth of the market, many alternative cloud services and their underlying systems are currently vying for the attention of cloud users and providers. Thus, benchmarking them is needed, to enable cloud users to make an informed choice, and to enable system DevOps to tune, design, and evaluate their systems. This requires focusing on old and new system properties, possibly leading to the re-design of classic benchmarking metrics, such as expressing performance as throughput and latency (response time), and the design of new, cloud-specic metrics. Addressing this requirement, in this work we focus on four system properties:

(i) elasticity of the cloud service, to accommodate large variations in the amount of service requested,
(ii) performance isolation between the tenants of shared cloud systems,
(iii) availability of cloud services and systems, and the
(iv) operational risk of running a production system in a cloud environment.

Focusing on key metrics, for each of these properties we review the state-of-the-art, then select or propose new metrics together with measurement approaches. We see the presented metrics as a foundation towards upcoming, industry-standard, cloud benchmarks.

Nikolas Herbst, Rouven Krebs, Giorgos Oikonomou, George Kousiouris, Athanasia Evangelinou, Alexandru Iosup, and Samuel Kounev. Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics. Technical Report SPEC-RG-2016-01 v.1.0, SPEC Research Group - Cloud Working Group, March 2016

The technical report can be found here.