Wednesday, 13 December 2017

Session 9: Distributed Systems Performance II

Understanding, Modelling, and Improving the Performance of Web Applications in Multicore Virtualised Environments

Authors:

Xi Chen (Imperial College London)
Chin Pang Ho (Imperial College London)
Rasha Osman (Imperial College London)
Peter G. Harrison (Imperial College London)
William J. Knottenbelt (Imperial College London)

Abstract:

As the computing industry enters the Cloud era, multicore architectures and virtualisation technologies are replacing traditional IT infrastructures. However, the complex relationship between applications and system resources in multicore virtualised environments is not well understood. Workloads such as web services and on-line financial applications have the requirement of high performance but benchmark analysis suggests that these applications do not optimally benefit from a higher number of cores. In this paper, we try to understand the scalability behaviour of network/CPU intensive applications running on multicore architectures. We begin by benchmarking the Petstore web application, noting the systematic imbalance that arises with respect to per-core workload. Having identified the reason for this phenomenon, we propose a queueing model which, when appropriately parametrised, reflects the trend in our benchmark results for up to 8 cores. Key to our approach is providing a fine-grained model which incorporates the idiosyncrasies of the operating system and the multiple CPU cores. Analysis of the model suggests a straightforward way to mitigate the observed bottleneck, which can be practically realised by the deployment of multiple virtual NICs within our VM. Next we make blind predictions to forecast performance with multiple virtual NICs. The validation results show that the model is able to predict the expected performance with relative errors ranging between 8 and 26%.

DOI: 10.1145/2568088.2568102

Full text: PDF

[#][]

An Evaluation of ZooKeeper for High Availability in System S

Authors:

Cuong M. Pham (University of Illinois at Urbana-Champaign)
Victor Dogaru (IBM Corp.)
Rohit Wagle (IBM T.J. Watson Research Center)
Chitra Venkatramani (IBM T.J. Watson Research Center)
Zbigniew Kalbarczyk (University of Illinois at Urbana-Champaign)
Ravishankar K. Iyer (University of Illinois at Urbana-Champaign)

Abstract:

ZooKeeper provides scalable, highly available coordination services for distributed applications. In this paper, we evaluate the use of ZooKeeper in a distributed stream computing system called System S to provide a resilient name service, dynamic configuration management, and system state management. The evaluation shed light on the advantages of using ZooKeeper in these contexts as well as its limitations. We also describe design changes we made to handle named objects in System S to overcome the limitations. We present detailed experimental results, which we believe will be beneficial to the community.

DOI: 10.1145/2568088.2576801

Full text: PDF

[#][]

Scalable Hybrid Stream and Hadoop Network Analysis System

Authors:

Vernon K. C. Bumgardner (University of Kentucky)
Victor W. Marek (University of Kentucky)

Abstract:

Collections of network traces have long been used in network traffic analysis. Flow analysis can be used in network anomaly discovery, intrusion detection and more generally, discovery of actionable events on the network. The data collected during processing may be also used for prediction and avoidance of traffic congestion, network capacity planning, and the development of software-defined networking rules. As network flow rates increase and new network technologies are introduced on existing hardware platforms, many organizations find themselves either technically or financially unable to generate, collect, and/or analyze network flow data. The continued rapid growth of network trace data, requires new methods of scalable data collection and analysis. We report on our deployment of a system designed and implemented at the University of Kentucky that supports analysis of network traffic across the enterprise. Our system addresses problems of scale in existing systems, by using distributed computing methodologies, and is based on a combination of stream and batch processing techniques. In addition to collection, stream processing using Storm is utilized to enrich the data stream with ephemeral environment data. Enriched stream-data is then used for event detection and near real-time flow analysis by an in-line complex event processor. Batch processing is performed by the Hadoop MapReduce framework, from data stored in HBase BigTable storage. In benchmarks on our 10 node cluster, using actual network data, we were able to stream process over 315k flows/sec. In batch analysis were we able to process over 2.6M flows/sec with a storage compression ratio of 6.7:1.

DOI: 10.1145/2568088.2568103

Full text: PDF

[#][]