Skip to content

Research at St Andrews

Plug and Play Bench: simplifying big data benchmarking using containers

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Author(s)

School/Research organisations

Abstract

The recent boom of big data, coupled with the challenges of its processing and storage gave rise to the development of distributed data processing and storage paradigms like MapReduce, Spark, and NoSQL databases. With the advent of cloud computing, processing and storing such massive datasets on clusters of machines is now feasible with ease. However, there are limited tools and approaches, which users can rely on to gauge and comprehend the performance of their big data applications deployed locally on clusters, or in the cloud. Researchers have started exploring this area by providing benchmarking suites suitable for big data applications. However, many of these tools are fragmented, complex to deploy and manage, and do not provide transparency with respect to the monetary cost of benchmarking an application. In this paper, we present Plug And Play Bench (PAPB1): aninfrastructure aware abstraction built to integrate and simplifythe deployment of big data benchmarking tools on clusters of machines. PAPB automates the tedious process of installing, configuring and executing common big data benchmark work-loads by containerising the tools and settings based on the underlying cluster deployment framework. Our proof of concept implementation utilises HiBench as the benchmark suite, HDP as the cluster deployment framework and Azure as the cloud platform. The paper further illustrates the inclusion of cost metrics based on the underlying Microsoft Azure cloud platform.
Close

Details

Original languageEnglish
Title of host publicationProceedings 2017 IEEE International Conference on Big Data (IEEE BigData 2017)
EditorsJian-Yun Nie, Zoran Obradovic, Toyotaro Suzumura, Rumi Ghosh, Raghunath Nambiar, Chonggang Wang, Hui Zang, Ricardo Baeza-Yates, Xiaohua Hu, Jeremy Kepner, Alfredo Cuzzocrea, Jian Tang, Masashi Toyoda
PublisherIEEE Computer Society
Pages2821-2828
Number of pages8
ISBN (Electronic)9781538627150
DOIs
Publication statusPublished - 11 Dec 2017
EventWorkshop on Benchmarking, Performance Tuning and Optimization for Big Data Applications (BPOD) - Boston, United States
Duration: 11 Dec 201714 Dec 2017
https://userpages.umbc.edu/~jianwu/BPOD-2017/

Workshop

WorkshopWorkshop on Benchmarking, Performance Tuning and Optimization for Big Data Applications (BPOD)
Abbreviated titleBPOD
CountryUnited States
CityBoston
Period11/12/1714/12/17
Internet address

Discover related content
Find related publications, people, projects and more using interactive charts.

View graph of relations

Related by author

  1. Cloud futurology

    Varghese, B., Leitner, P., Ray, S., Chard, K., Barker, A., Elkhatib, Y., Herry, H., Hong, C., Singer, J., Tso, F. P., Yoneki, E. & Zhani, M., Sep 2019, In : IEEE Computer. 52, 9, p. 68-77 10 p.

    Research output: Contribution to journalArticle

  2. Cloud benchmarking for maximising performance of scientific applications

    Varghese, B., Akgun, O., Miguel, I. J., Thai, L. T. & Barker, A. D., 1 Jan 2019, In : IEEE Transactions on Cloud Computing. 7, 1, p. 170-182 13 p., 7553491.

    Research output: Contribution to journalArticle

  3. Algorithms for optimising heterogeneous Cloud virtual machine clusters

    Thai, L. T., Varghese, B. & Barker, A. D., 12 Dec 2016, 2016 IEEE International Conference on Cloud Computing Technology and Science. IEEE, p. 118-125 8 p. 7830674

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

  4. Minimising the execution of unknown Bag-of-Task jobs with deadlines on the Cloud

    Thai, L. T., Varghese, B. & Barker, A. D., 1 Jun 2016, DIDC '16 Proceedings of the ACM International Workshop on Data-Intensive Distributed Computing. ACM, p. 3-10

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

ID: 251730530

Top