<abstract xmlns="http://www.w3.org/1999/xhtml"><p> Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such as locality of data, user share fairness, and resource awareness. This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave nodes. The experiments include variation in data sizes, use of two different data processing applications, and variation in the number of nodes used in processing. The article analyzes the performance of the job scheduling algorithms based on various relevant performance measures. The results of the experiments are evident of the performance being affected by the job scheduling parameters, the type of applications, the number of nodes in the cluster, and size of the input data.</p></abstract>

Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such as locality of data, user share fairness, and resource awareness. This article focuses on empirically evaluating the performance of three schedulers: First In First Out (FIFO), Fair scheduler, and Capacity scheduler. To carry out the experimental evaluation, we implement our own Hadoop cluster testbed, consisting of four machines, in which one of the machines works as the master node and all four machines work as slave nodes. The experiments include variation in data sizes, use of two different data processing applications, and variation in the number of nodes used in processing. The article analyzes the performance of the job scheduling algorithms based on various relevant performance measures. The results of the experiments are evident of the performance being affected by the job scheduling parameters, the type of applications, the number of nodes in the cluster, and size of the input data.

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Department of Information Technology, Dharmsinh Desai University, Nadiad,

Institute of Engineering & Technology, Ahmedabad University, Ahmedabad,

Cybernetics and Information Technologies

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce"}

Several Job scheduling algorithms have been developed for Hadoop-Map Reduce model, which vary widely in design and behavior for handling different issues such...

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Publicado en línea: 06 abr 2017

Páginas: 146 - 163

DOI: https://doi.org/10.1515/cait-2017-0012

Palabras clave
Big Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation

© by Jyoti V. Gautam

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Empirical Study of Job Scheduling Algorithms in Hadoop MapReduce

Publicado en línea: 06 abr 2017

Páginas: 146 - 163

DOI: https://doi.org/10.1515/cait-2017-0012

Palabras claveBig Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation

© by Jyoti V. Gautam

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Palabras clave
Big Data, Hadoop, Map Reduce, job scheduling, analysis, experimental evaluation