Mission Statement

The mission of the Cloud Working Group of SPEC RG is to facilitate and foster leading research in cloud computing through experience sharing, constructive feedback, agreement on definitions, specification of metrics, and dissemination of novel methods, techniques and tools for quantitative evaluation and analysis of cloud-based systems.

General Topics of Interest

Topics of Interest include, but are not limited to, the following:

  • Methods and metrics for quantitative evaluation of cloud systems
    • Classical performance metrics, including response time, throughput, and efficiency
    • Elasticity
    • Performance isolation
    • Scalability
    • Dependability (with focus on availability, reliability and resilience)
    • Power consumption and energy efficiency
  • Emerging techniques
    • Digital twinning, which aims to create a digital copy of the real-world processes, services, and devices, and, combined with monitoring, benchmarking, and analysis capabilities, enhances the observability of the system under test and enables new modes of what-if experimentation
    • Advanced use of AI in systems operations, in particular, for quantitative evaluation and modeling
  • Monitoring methodologies
  • Benchmarking methodologies
  • Modeling and simulation methodologies
  • Performance prediction methodologies
  • Performance evaluation methodologies
  • Workload characterization and reproduction
  • Provisioning and monitoring
  • Resource management and optimization of complex workloads and operations, for example, of scientific workflows

Activities

Community Activities:

  • Organizing the yearly in-person meeting of the Cloud Group
  • Organizing the monthly online meeting of the Cloud Group
  • Contributing to the yearly SPEC RG Newsletter
  • Organizing the HotCloudPerf international workshop on hot topics in cloud performance engineering (with ICPE)
  • Organizing various national- and international-scale events, for example, Dagstuhl Seminars on key topics
  • Publishing / releasing Open Science artifacts, in particular, open-source software and FAIR data

Current Activities:

  • Reviewing existing research literature on cloud benchmarking
  • Preparing a survey overview of cloud usage patterns and application scenarios
  • Providing feedback on works in progress
  • Maintaining a repository of tools for cloud evaluation
  • Developing benchmark methodologies for the cloud with current focus on cloud elasticity and performance isolation
  • Digital twinning of ICT infrastructure
  • Digital continuum operations – where datacenters and HPC sites, edge deployments, and end-point devices can seamlessly network and interoperate
  • Resource management and optimization around (scientific) workflows
  • Defining terminologies

Potential Future Activities (based on interest):

  • Leveraging an integrated and shared, PlanetLab-like facility for cloud benchmarking
  • Archives and live collections of workload and operational traces
  • A repository of use cases