Highlighted Publications

Selected joint member publications as well as technical reports published by SPEC RG are available at the publication page.

Group Publications

In the following, we list a selection of relevant publications by members of the RG DevOps Performance working group.

• Simon Eismann, Diego Elias Costa, Lizhi Liao, Cor-Paul Bezemer, Weiyi Shang, André van Hoorn, Samuel Kounev.
  A case study on the stability of performance tests for serverless applications. J. Syst. Softw., volume 189, pp. 111294. 2022.
  DOI: 10.1016/j.jss.2022.111294
• Vittorio Cortellessa, Daniele Di Pompeo, Romina Eramo, Michele Tucci.
  A model-driven approach for continuous performance engineering in microservice-based systems. J. Syst. Softw., volume 183, pp. 111084. 2022.
  DOI: 10.1016/j.jss.2021.111084
• Henning Schulz, Dusan Okanovic, André van Hoorn, Petr Tuma.
  Context-tailored Workload Model Generation for Continuous Representative Load Testing. ICPE '21: ACM/SPEC International Conference on Performance Engineering, Virtual Event, France, April 19-21, pp. 21-32. 2021.
  DOI: 10.1145/3427921.3450240
• Diego Costa, Cor-Paul Bezemer, Philipp Leitner, Artur Andrzejak.
  What's Wrong with My Benchmark Results? Studying Bad Practices in JMH Benchmarks. IEEE Trans. Software Eng., volume 47, pp. 1452-1467. 2021.
  DOI: 10.1109/TSE.2019.2925345
• Alim Ul Gias, André van Hoorn, Lulai Zhu, Giuliano Casale, Thomas F. Düllmann, Michael Wurster.
  Performance Engineering for Microservices and Serverless Applications: The RADON Approach. Companion of the ACM/SPEC International Conference on Performance Engineering, ICPE, Edmonton, AB, Canada, April 20-24, pp. 46-49. 2020.
  DOI: 10.1145/3375555.3383120
• Simon Eismann, Cor-Paul Bezemer, Weiyi Shang, Dusan Okanovic, André van Hoorn.
  Microservices: A Performance Tester's Dream or Nightmare?. ICPE '20: ACM/SPEC International Conference on Performance Engineering, Edmonton, AB, Canada, April 20-24, pp. 138-149. 2020.
  DOI: 10.1145/3358960.3379124
• Manar Mazkatli, David Monschein, Johannes Grohmann, Anne Koziolek.
  Incremental Calibration of Architectural Performance Models with Parametric Dependencies. 2020 IEEE International Conference on Software Architecture, ICSA, Salvador, Brazil, March 16-20, pp. 23-34. 2020.
  DOI: 10.1109/ICSA47634.2020.00011
• Sara Gholami, Alireza Goli, Cor-Paul Bezemer, Hamzeh Khazaei.
  A Framework for Satisfying the Performance Requirements of Containerized Software Systems Through Multi-Versioning. ICPE '20: ACM/SPEC International Conference on Performance Engineering, Edmonton, AB, Canada, April 20-24, pp. 150-160. 2020.
  DOI: 10.1145/3358960.3379125
• Sonya Voneva, Manar Mazkatli, Johannes Grohmann, Anne Koziolek.
  Optimizing Parametric Dependencies for Incremental Performance Model Extraction. Software Architecture - 14th European Conference, ECSA Tracks and Workshops, L'Aquila, Italy, September 14-18, Proceedings, pp. 228-240. 2020.
  DOI: 10.1007/978-3-030-59155-7_17
• Henning Schulz, Dusan Okanovic, André van Hoorn, Vincenzo Ferme, Cesare Pautasso.
  Behavior-driven Load Testing Using Contextual Knowledge - Approach and Experiences. Proceedings of the ACM/SPEC International Conference on Performance Engineering, ICPE, Mumbai, India, April 7-11, pp. 265-272. 2019.
  DOI: 10.1145/3297663.3309674
• Shahnaz M. Shariff, Heng Li, Cor-Paul Bezemer, Ahmed E. Hassan, Thanh H. D. Nguyen, Parminder Flora.
  Improving the testing efficiency of selenium-based load tests. Proceedings of the 14th International Workshop on Automation of Software Test, AST@ICSE, May 27, Montreal, QC, Canada, pp. 14-20. 2019.
  DOI: 10.1109/AST.2019.00008
• Cor-Paul Bezemer, Simon Eismann, Vincenzo Ferme, Johannes Grohmann, Robert Heinrich, Pooyan Jamshidi, Weiyi Shang, André van Hoorn, Mónica Villavicencio, Jürgen Walter, Felix Willnecker.
  How is Performance Addressed in DevOps?. Proceedings of the ACM/SPEC International Conference on Performance Engineering, ICPE, Mumbai, India, April 7-11, pp. 45-50. 2019.
  DOI: 10.1145/3297663.3309672
• Suhas Kabinna, Cor-Paul Bezemer, Weiyi Shang, Mark D. Syer, Ahmed E. Hassan.
  Examining the stability of logging statements. Empir. Softw. Eng., volume 23, pp. 290-333. 2018.
  DOI: 10.1007/s10664-017-9518-0
• Simon Eismann, Jürgen Walter, Jóakim von Kistowski, Samuel Kounev.
  Modeling of Parametric Dependencies for Performance Prediction of Component-Based Software Systems at Run-Time. IEEE International Conference on Software Architecture, ICSA, Seattle, WA, USA, April 30 - May 4, pp. 135-144. 2018.
  DOI: 10.1109/ICSA.2018.00023
• Christian Vögele, André van Hoorn, Eike Schulz, Wilhelm Hasselbring, Helmut Krcmar.
  WESSBAS: extraction of probabilistic workload specifications for load testing and performance prediction - a model-driven approach for session-based application systems. Softw. Syst. Model., volume 17, pp. 443-477. 2018.
  DOI: 10.1007/s10270-016-0566-5
• Thomas F. Düllmann, Robert Heinrich, André van Hoorn, Teerat Pitakrat, Jürgen Walter, Felix Willnecker.
  CASPA: A Platform for Comparability of Architecture-Based Software Performance Engineering Approaches. 2017 IEEE International Conference on Software Architecture Workshops, ICSA Workshops, Gothenburg, Sweden, April 5-7, pp. 294-297. 2017.
  DOI: 10.1109/ICSAW.2017.26
• Robert Heinrich, André van Hoorn, Holger Knoche, Fei Li, Lucy Ellen Lwakatare, Claus Pahl, Stefan Schulte, Johannes Wettinger.
  Performance Engineering for Microservices: Research Challenges and Directions. Companion Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering, ICPE, L'Aquila, Italy, April 22-26, pp. 223-226. 2017.
  DOI: 10.1145/3053600.3053653
• Jürgen Walter, André van Hoorn, Samuel Kounev.
  Automated and Adaptable Decision Support for Software Performance Engineering. Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools, VALUETOOLS, Venice, Italy, December 05-07, pp. 66-73. 2017.
  DOI: 10.1145/3150928.3150952
• Philipp Leitner, Cor-Paul Bezemer.
  An Exploratory Study of the State of Practice of Performance Testing in Java-Based Open Source Projects. Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering, ICPE, L'Aquila, Italy, April 22-26, pp. 373-384. 2017.
  DOI: 10.1145/3030207.3030213
• Jürgen Walter, Dusan Okanovic, Samuel Kounev.
  Mapping of Service Level Objectives to Performance Queries. Companion Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering, ICPE, L'Aquila, Italy, April 22-26, pp. 197-202. 2017.
  DOI: 10.1145/3053600.3053646
• Ravjot Singh, Cor-Paul Bezemer, Weiyi Shang, Ahmed E. Hassan.
  Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm. Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE, Delft, The Netherlands, March 12-16, pp. 309-320. 2016.
  DOI: 10.1145/2851553.2851576
• Suhas Kabinna, Cor-Paul Bezemer, Weiyi Shang, Ahmed E. Hassan.
  Logging library migrations: a case study for the apache software foundation projects. Proceedings of the 13th International Conference on Mining Software Repositories, MSR, Austin, TX, USA, May 14-22, pp. 154-164. 2016.
  DOI: 10.1145/2901739.2901769
• Tarek M. Ahmed, Cor-Paul Bezemer, Tse-Hsun Chen, Ahmed E. Hassan, Weiyi Shang.
  Studying the effectiveness of application performance management (APM) tools for detecting performance regressions for web applications: an experience report. Proceedings of the 13th International Conference on Mining Software Repositories, MSR, Austin, TX, USA, May 14-22, pp. 1-12. 2016.
  DOI: 10.1145/2901739.2901774
• Dusan Okanovic, André van Hoorn, Christoph Heger, Alexander Wert, Stefan Siegl.
  Towards Performance Tooling Interoperability: An Open Format for Representing Execution Traces. Computer Performance Engineering - 13th European Workshop, EPEW, Chios, Greece, October 5-7, Proceedings, pp. 94-108. 2016.
  DOI: 10.1007/978-3-319-46433-6_7
• Suhas Kabinna, Weiyi Shang, Cor-Paul Bezemer, Ahmed E. Hassan.
  Examining the Stability of Logging Statements. IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, SANER, Suita, Osaka, Japan, March 14-18 - Volume 1, pp. 326-337. 2016.
  DOI: 10.1109/SANER.2016.29
• Jürgen Walter, André van Hoorn, Heiko Koziolek, Dusan Okanovic, Samuel Kounev.
  Asking "What"?, Automating the "How"?: The Vision of Declarative Performance Engineering. Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE, Delft, The Netherlands, March 12-16, pp. 91-94. 2016.
  DOI: 10.1145/2851553.2858662
• Andreas Brunnert, André van Hoorn, Felix Willnecker, Alexandru Danciu, Wilhelm Hasselbring, Christoph Heger, Nikolas Herbst, Pooyan Jamshidi, Reiner Jung, Jóakim von Kistowski, Anne Koziolek, Johannes Kroß, Simon Spinner, Christian Vögele, Jürgen Walter, Alexander Wert.
  Performance-oriented DevOps: A Research Agenda. Technical Report SPEC-RG-2015-01, SPEC Research Group - DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC). 2015.
  http://arxiv.org/abs/1508.04752

Reports of co-organized events:

• Fabian Beck, Alexandre Bergel, Cor-Paul Bezemer, Katherine E. Isaacs.
  Visualizing systems and software performance. Report on the GI-Dagstuhl seminar for young researchers, July 9-13. 2018.
  DOI: 10.7287/peerj.preprints.27253v1
• André van Hoorn, Pooyan Jamshidi, Philipp Leitner, Ingo Weber.
  Software Performance Engineering in the DevOps World. Report from GI-Dagstuhl Seminar 16394. 2017.
  http://arxiv.org/abs/1709.08951