Accelerating MapReduce framework on multi-GPU systems

Hai Jiang, Yi Chen, Zhi Qiao, Kuan Ching Li, Won Woo Ro, Jean Luc Gaudiot

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Graphics processors evolve rapidly and promise to support power-efficient, cost, differentiated price-performance, and scalable high performance computing. MapReduce is a well-known distributed programming model to ease the development of applications for large-scale data processing on a large number of commodity CPUs. When compared to CPUs, GPUs are an order of magnitude faster in terms of computation power and memory bandwidth, but they are harder to program. Although several studies have implemented the MapReduce model on GPUs, most of them are based on the single GPU model and bounded by a GPU memory with inefficient atomic operations. This paper focuses on the development of MGMR, a standalone MapReduce system that utilizes multiple GPUs to manage large-scale data processing beyond the GPU memory limitation, and also to eliminate serial atomic operations. Experimental results have demonstrated the effectiveness of MGMR in handling large data sets.

Original languageEnglish
Pages (from-to)293-301
Number of pages9
JournalCluster Computing
Volume17
Issue number2
DOIs
Publication statusPublished - 2014 Jan 1

Fingerprint

Data storage equipment
Program processors
Graphics processing unit
Bandwidth
Costs

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications

Cite this

Jiang, H., Chen, Y., Qiao, Z., Li, K. C., Ro, W. W., & Gaudiot, J. L. (2014). Accelerating MapReduce framework on multi-GPU systems. Cluster Computing, 17(2), 293-301. https://doi.org/10.1007/s10586-013-0276-5
Jiang, Hai ; Chen, Yi ; Qiao, Zhi ; Li, Kuan Ching ; Ro, Won Woo ; Gaudiot, Jean Luc. / Accelerating MapReduce framework on multi-GPU systems. In: Cluster Computing. 2014 ; Vol. 17, No. 2. pp. 293-301.
@article{650299346e97404eb50c80d59de32400,
title = "Accelerating MapReduce framework on multi-GPU systems",
abstract = "Graphics processors evolve rapidly and promise to support power-efficient, cost, differentiated price-performance, and scalable high performance computing. MapReduce is a well-known distributed programming model to ease the development of applications for large-scale data processing on a large number of commodity CPUs. When compared to CPUs, GPUs are an order of magnitude faster in terms of computation power and memory bandwidth, but they are harder to program. Although several studies have implemented the MapReduce model on GPUs, most of them are based on the single GPU model and bounded by a GPU memory with inefficient atomic operations. This paper focuses on the development of MGMR, a standalone MapReduce system that utilizes multiple GPUs to manage large-scale data processing beyond the GPU memory limitation, and also to eliminate serial atomic operations. Experimental results have demonstrated the effectiveness of MGMR in handling large data sets.",
author = "Hai Jiang and Yi Chen and Zhi Qiao and Li, {Kuan Ching} and Ro, {Won Woo} and Gaudiot, {Jean Luc}",
year = "2014",
month = "1",
day = "1",
doi = "10.1007/s10586-013-0276-5",
language = "English",
volume = "17",
pages = "293--301",
journal = "Cluster Computing",
issn = "1386-7857",
publisher = "Kluwer Academic Publishers",
number = "2",

}

Jiang, H, Chen, Y, Qiao, Z, Li, KC, Ro, WW & Gaudiot, JL 2014, 'Accelerating MapReduce framework on multi-GPU systems', Cluster Computing, vol. 17, no. 2, pp. 293-301. https://doi.org/10.1007/s10586-013-0276-5

Accelerating MapReduce framework on multi-GPU systems. / Jiang, Hai; Chen, Yi; Qiao, Zhi; Li, Kuan Ching; Ro, Won Woo; Gaudiot, Jean Luc.

In: Cluster Computing, Vol. 17, No. 2, 01.01.2014, p. 293-301.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Accelerating MapReduce framework on multi-GPU systems

AU - Jiang, Hai

AU - Chen, Yi

AU - Qiao, Zhi

AU - Li, Kuan Ching

AU - Ro, Won Woo

AU - Gaudiot, Jean Luc

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Graphics processors evolve rapidly and promise to support power-efficient, cost, differentiated price-performance, and scalable high performance computing. MapReduce is a well-known distributed programming model to ease the development of applications for large-scale data processing on a large number of commodity CPUs. When compared to CPUs, GPUs are an order of magnitude faster in terms of computation power and memory bandwidth, but they are harder to program. Although several studies have implemented the MapReduce model on GPUs, most of them are based on the single GPU model and bounded by a GPU memory with inefficient atomic operations. This paper focuses on the development of MGMR, a standalone MapReduce system that utilizes multiple GPUs to manage large-scale data processing beyond the GPU memory limitation, and also to eliminate serial atomic operations. Experimental results have demonstrated the effectiveness of MGMR in handling large data sets.

AB - Graphics processors evolve rapidly and promise to support power-efficient, cost, differentiated price-performance, and scalable high performance computing. MapReduce is a well-known distributed programming model to ease the development of applications for large-scale data processing on a large number of commodity CPUs. When compared to CPUs, GPUs are an order of magnitude faster in terms of computation power and memory bandwidth, but they are harder to program. Although several studies have implemented the MapReduce model on GPUs, most of them are based on the single GPU model and bounded by a GPU memory with inefficient atomic operations. This paper focuses on the development of MGMR, a standalone MapReduce system that utilizes multiple GPUs to manage large-scale data processing beyond the GPU memory limitation, and also to eliminate serial atomic operations. Experimental results have demonstrated the effectiveness of MGMR in handling large data sets.

UR - http://www.scopus.com/inward/record.url?scp=84901592469&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901592469&partnerID=8YFLogxK

U2 - 10.1007/s10586-013-0276-5

DO - 10.1007/s10586-013-0276-5

M3 - Article

AN - SCOPUS:84901592469

VL - 17

SP - 293

EP - 301

JO - Cluster Computing

JF - Cluster Computing

SN - 1386-7857

IS - 2

ER -