Tuesday, 12 December 2017

Session 9: Performance Modeling and Prediction

Stream-Based Event Prediction Using Bayesian and Bloom Filters

Authors:

Miao Wang (University College Dublin)
Viliam Holub (University College Dublin)
John Murphy (University College Dublin)
Patrick O'Sullivan (IBM Software Group)

Abstract:

Nowadays, enterprise software systems store a large amount of operational information in logs. Manually analysing these data can be time-consuming and error-prone. Although a static knowledge database eases the task to capture recurring problems, maintaining such a knowledge repository requires periodic knowledge updates by domain experts. Moreover, as the repository grows, the problem of memory efficiency will also arise.

Our goal is to enable administrators to efficiently capture interesting data in a high volume stream of events in real-time. We are proposing a statistical approach for software applications to be automatically trained with a smaller dataset to efficiently predict interesting data under such conditions. The proposed solution maintains a stable memory usage by migrating keywords from a dynamic data structure to fixed sized data structures (Bloom Filter). In particular, the solution has achieved better prediction results by enhancing the Bayesian theory with belief modifiers.

DOI: 10.1145/2479871.2479903

Full text: PDF

[#][]

On-Line Fair Allocations Based on Bottlenecks and Global Priorities

Authors:

Yoel Zeldes (The Hebrew University)
Dror G. Feitelson (The Hebrew University)

Abstract:

System bottlenecks, namely those resources which are subjected to high contention, constrain system performance. Hence effective resource management should be done by focusing on the bottleneck resources and allocating them to the most deserving clients. It has been shown that for any combination of entitlements and requests a fair allocation of bottleneck resources can be found, using an off-line algorithm that is given full information in advance regarding the needs of each client. We extend this result to the on-line case with no prior information. To this end we introduce a simple greedy algorithm. In essence, when a scheduling decision needs to be made, this algorithm selects the client that has the largest minimal gap between its entitlement and its current allocation among all the bottleneck resources. Importantly, this algorithm takes a global view of the system, and assigns each client a single priority based on his usage of all the resources; this single priority is then used to make coordinated scheduling decisions on all the resources. Extensive simulations show that this algorithm achieves fair allocations according to the desired entitlements for a wide range of conditions, without using any prior information regarding resource requirements. It also follows shifting usage patterns, including situations where the bottlenecks change with time.

DOI: 10.1145/2479871.2479904

Full text: PDF

[#][]

Survivability Models for the Assessment of Smart Grid Distribution Automation Network Designs

Authors:

Alberto Avritzer (Siemens Corporation)
Sindhu Suresh (Siemens Corporation)
Daniel Sadoc Menasché (Federal University of Rio de Janeiro)
Rosa Maria Meri Leão (Federal University of Rio de Janeiro)
Edmundo de Souza e Silva (Federal University of Rio de Janeiro)
Morganna Carmem Diniz (Federal University of the State of Rio de Janeiro)
Kishor Trivedi (Duke University)
Lucia Happe (Karlsruhe Institute of Technology)
Anne Koziolek (University of Zurich)

Abstract:

Smart grids are fostering a paradigm shift in the realm of power distribution systems. Whereas traditionally different components of the power distribution system have been provided and analyzed by different teams through different lenses, smart grids require a unified and holistic approach that takes into consideration the interplay of communication reliability, energy backup, distribution automation topology, energy storage and intelligent features such as automated failure detection, isolation and restoration (FDIR) and demand response.

In this paper, we present an analytical model and metrics for the survivability assessment of the distribution power grid network. The proposed metrics extend the system average interruption duration index (SAIDI), accounting for the fact that after a failure the energy demand and supply will vary over time during a multi-step recovery process. The analytical model used to compute the proposed metrics is built on top of three design principles: state space factorization, state aggregation and initial state conditioning. Using these principles, we reduce a Markov chain model with large state space cardinality to a set of much simpler models that are amenable to analytical treatment and efficient numerical solution. In the special case where demand response is not integrated with FDIR, we provide closed form solutions to the metrics of interest, such as the mean time to repair a given set of sections.

We have evaluated the presented model using data from a real power distribution grid and we have found that survivability of distribution power grids can be improved by the integration of the demand response feature with automated FDIR approaches. Our empirical results indicate the importance of quantifying survivability to support investment decisions at different parts of the power grid distribution network.

DOI: 10.1145/2479871.2479905

Full text: PDF

[#][]

Benchmarking Approach for Designing a MapReduce Performance Model

Authors:

Zhuoyao Zhang (University of Pennsylvania)
Ludmila Cherkasova (Hewlett-Packard Labs)
Boon Thau Loo (University of Pennsylvania)

Abstract:

In MapReduce environments, many of the programs are reused for processing a regularly incoming new data. A typical user question is how to estimate the completion time of these programs as a function of a new dataset and the cluster resources. In this work1, we offer a novel performance evaluation framework for answering this question. We observe that the execution of each map (reduce) tasks consists of specific, well-defined data processing phases. Only map and reduce functions are custom and their executions are user-defined for different MapReduce jobs. The executions of the remaining phases are generic and depend on the amount of data processed by the phase and the performance of underlying Hadoop cluster. First, we design a set of parameterizable microbenchmarks to measure generic phases and to derive a platform performance model of a given Hadoop cluster. Then using the job past executions, we summarize job’s properties and performance of its custom map/reduce functions in a compact job profile. Finally, by combining the knowledge of the job profile and the derived platform performance model, we offer a MapReduce performance model that estimates the program completion time for processing a new dataset. The evaluation study justifies our approach and the proposed framework: we are able to accurately predict performance of the diverse set of twelve MapReduce applications. The predicted completion times for most experiments are within 10% of the measured ones (with a worst case resulting in 17% of error) on our 66-node Hadoop cluster.

DOI: 10.1145/2479871.2479906

Full text: PDF

[#][]