Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics

Nikolas Herbst, Rouven Krebs, Giorgos Oikonomou, George Kousiouris, Athanasia Evangelinou, Alexandru Iosup, Samuel Kounev. Technical Report SPEC-RG-2016-01, SPEC Research Group - Cloud Working Group, Standard Performance Evaluation Corporation (SPEC). March 2016.

(i) elasticity of the cloud service, to accommodate large variations in the amount of service requested, (ii) performance isolation between the tenants of shared cloud systems, (iii) availability of cloud services and systems, and the (iv) operational risk of running a production system in a cloud environment.

Focusing on key metrics, for each of these properties we review the state-of-the-art, then select or propose new metrics together with measurement approaches. We see the presented metrics as a foundation towards upcoming, industry-standard, cloud benchmarks.

Keywords: Cloud Computing; Metrics; Measurement; Benchmarking; Elasticity; Isolation; Performance; Service Level Objective; Availability; Operational Risk.

BibTex entry

@TechReport{SPEC-RG-2016-01-CloudMetrics,
Title                    = {Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics},
Author                   = {Nikolas Herbst and Rouven Krebs and Giorgos Oikonomou and George Kousiouris and Athanasia Evangelinou and Alexandru Iosup and Samuel Kounev},
Institution              = {SPEC Research Group --- Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)},
Year                     = {2016},
Month                    = march,
Number                   = {SPEC-RG-2016-01},
Url                      = {https://research.spec.org/fileadmin/user_upload/documents/rg_cloud/endorsed_publications/SPEC-RG-2016-01_CloudMetrics.pdf}
}