Hadoop MapReduce Performance Study on ARM cluster

Yanjun Wangyanjun@cse.unl.edu

Outlines

• Motivation• Introduction• Evaluation• Conclusion• Questions

Motivation

• A credit card size Raspberry Pi can run general Linux with very low power consumption

Motivation

• ARM cluster vs. x86_64 cluster

Introduction

• Hadoop MapReduce• Cubieboard2

Evaluation• Environment– ARM cluster: 4 cubieboard2, 1 head node, 3 worker

nodes; lubuntu for ARM, java-1.7 for ARM– X86_64 cluster: 2 firefly nodes. 1 head node, 1 worker

node; CentOS 6.3, java-1.7• Hadoop 1.2 [1]• Testcases– Loadgen– MDAD (Molecular Dynamics Simulation based on

Hadoop MapReduce [2])

• [1]Apache Hadoop• [2]Chen He, “Molecular Dynamics Simulation based on Hadoop MapReduce” , Master thesis, 2011

Evaluation

• Equivalent Performance– Run program on current device and get

turnaround time– To achive the same turnaround time, how many

new devices we need, or could we this?

• Energy consumption– Kill-a-Watt device to collect ARM cluster energy;– ServerTech PDU for gathering x86_64 cluster

energy consumption

Evaluation

• loadgen

Evaluation

• MDAD

Evaluation

• Loadgen Energy

Evaluation

• MDAD energy

Conclusion

• We build a ARM cluster which is composed of 4 cubieboard2 cards.

• We setup Hadoop cluster on the ARM cluster• We compared the performance and energy

consumption between two clusters• Based on our current data, we conclude that

ARM cluster is not an alternative choice to replace X86_64.