Repository of Peer-Reviewed Tools Launched

Thursday, February 2, 2012 05:00

SPEC RG opens a repository of peer-reviewed tools for quantitative system evaluation and analysis.

A repository of peer-reviewed tools for quantitative system evaluation and analysis is now available at the SPEC RG website. As a contribution to the community, SPEC RG addresses the need for a collection of tools that have undergone a thorough review process by multiple independent experts to ensure high quality and relevance. The review process covers important quality factors, including maturity, availability and usability.

The repository is intended to have a broad scope, covering system evaluation and analysis with respect to both classical performance metrics, such as response time, throughput, scalability and efficiency as well as other non-functional system properties included under the term dependability, e.g., availability, reliability, and security. In particular, tools for measurement, profiling, workload characterization, load testing, stress testing and resilience testing are solicited.

Open And Growing

Among the first tools to pass the review process are Faban and Kieker. Faban is a framework to automate server performance tests in multi-tier environments, already used by the SPECjEnterprise2010 and SPECsipInfrastructure2011 benchmarks. The tool was originally developed at Sun Microsystems and is distributed under the CDDL 1.0 license. Kieker is a framework for monitoring and analysis of the performance of software systems. Kieker was developed by a research team from the University of Kiel and is distributed under an Apache 2.0 License.

More tools are currently under review and further submissions are encouraged. After a careful review process, the tools are published on the SPEC RG website. The tools are redistributed without modification and the submitter retains the associated rights and responsibilities. As part of the acceptance, SPEC RG may ask the authors to make changes or enhance certain features or aspects but SPEC RG itself does not make any changes.

Mature, Available, Usable/projects/tools.html