Saturday, 17 April 2021
Friday, 11. November 2016 06:38

Survey on Performance-aware DevOps Launched By SPEC RG

This survey is part of a larger project aiming on building best-practice infrastructures for Performance-aware DevOps.

Software developers, IT operators, and researchers know quite well how to handle performance in classic environments involving a strict separation between development (Dev) and operations (Ops). DevOps aims to merge these silos and aims to automate and accelerate the delivery and operations processes. Most of the time, performance does not receive enough or even any attention. In our opinion, this is primarily due to the fact that present performance management practices are not integrated and adapted to typical DevOps processes, especially in terms of automation and holistic tool support.

DevOps cycle


In our recent report “Performance-oriented DevOps: A Research Agenda”, we — the SPEC RG DevOps Performance Working Group — surveyed state-of-the-art performance management practices and identified challenges and opportunities for their integration into DevOps. The covered performance practices range from model-based and measurement-based approaches over anti-pattern detection to root cause analysis and optimization approaches. Since the number of approaches that are not easily adaptable to DevOps processes or focus on either development or operations considerably increased, we additionally outlined multiple challenges and activities that we think are necessary in order to support and adapt performance engineering activities during rapid iterations in DevOps.

In order to bring these challenges and opportunities into action and support delivering high-quality software, we are currently developing a blueprint for performance-aware DevOps processes and infrastructures. We plan to include artifacts, tools, as well as methods that shall serve as a reference architecture and framework for a holistic performance management throughout the whole software life-cycle.

To identify and capture the current state-of-the-art of performance practices, but also present problems and issues, we have launched a survey that we would like to promote and encourage you or your organization to participate in. We would like to find out how performance concerns are currently addressed in industrial DevOps practice and plan to integrate the impressions and results in our blueprint. Furthermore, we would like to know whether classical paradigms still dominate in your organization, when performance evaluations are conducted, what metrics are relevant for you, and what actions are applied after a performance evaluation.

The survey consists of multiple short questions and takes about 15 minutes. It specifically addresses performance methods, techniques, and tools that are being used in companies’ software delivery processes. It is of utmost importance to us, that you answer all questions, since it helps us to draw representative conclusions.

Our long-term aim is to not only conduct this survey once, but to benchmark the state-of-the-art continuously, compare the results over a longer period, and to regularly incorporate outcomes to our blueprint.

This survey is an activity of the DevOps Performance Working Group of the Standard Performance Evaluation Corporation’s Research Group (SPEC RG) as part of a larger project on building an open best practice reference infrastructure for performance-aware DevOps. The survey results will directly influence the reference infrastructure.


We kindly invite and encourage industry practitioners to participate in the online survey, which is available at