Cite Article

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Choose citation format

BibTeX

@article{IJASEIT9073,
   author = {Arwan A. Khoiruddin and Nordin Zakaria and Hitham Seddig Alhussian},
   title = {Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System},
   journal = {International Journal on Advanced Science, Engineering and Information Technology},
   volume = {10},
   number = {5},
   year = {2020},
   pages = {1913--1919},
   keywords = {Hadoop; scheduler; reduce task; asynchronous; non-blocking.},
   abstract = {Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.},
   issn = {2088-5334},
   publisher = {INSIGHT - Indonesian Society for Knowledge and Human Development},
   url = {http://ijaseit.insightsociety.org/index.php?option=com_content&view=article&id=9&Itemid=1&article_id=9073},
   doi = {10.18517/ijaseit.10.5.9073}
}

EndNote

%A Khoiruddin, Arwan A.
%A Zakaria, Nordin
%A Alhussian, Hitham Seddig
%D 2020
%T Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System
%B 2020
%9 Hadoop; scheduler; reduce task; asynchronous; non-blocking.
%! Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System
%K Hadoop; scheduler; reduce task; asynchronous; non-blocking.
%X Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.
%U http://ijaseit.insightsociety.org/index.php?option=com_content&view=article&id=9&Itemid=1&article_id=9073
%R doi:10.18517/ijaseit.10.5.9073
%J International Journal on Advanced Science, Engineering and Information Technology
%V 10
%N 5
%@ 2088-5334

IEEE

Arwan A. Khoiruddin,Nordin Zakaria and Hitham Seddig Alhussian,"Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System," International Journal on Advanced Science, Engineering and Information Technology, vol. 10, no. 5, pp. 1913-1919, 2020. [Online]. Available: http://dx.doi.org/10.18517/ijaseit.10.5.9073.

RefMan/ProCite (RIS)

TY  - JOUR
AU  - Khoiruddin, Arwan A.
AU  - Zakaria, Nordin
AU  - Alhussian, Hitham Seddig
PY  - 2020
TI  - Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System
JF  - International Journal on Advanced Science, Engineering and Information Technology; Vol. 10 (2020) No. 5
Y2  - 2020
SP  - 1913
EP  - 1919
SN  - 2088-5334
PB  - INSIGHT - Indonesian Society for Knowledge and Human Development
KW  - Hadoop; scheduler; reduce task; asynchronous; non-blocking.
N2  - Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.
UR  - http://ijaseit.insightsociety.org/index.php?option=com_content&view=article&id=9&Itemid=1&article_id=9073
DO  - 10.18517/ijaseit.10.5.9073

RefWorks

RT Journal Article
ID 9073
A1 Khoiruddin, Arwan A.
A1 Zakaria, Nordin
A1 Alhussian, Hitham Seddig
T1 Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System
JF International Journal on Advanced Science, Engineering and Information Technology
VO 10
IS 5
YR 2020
SP 1913
OP 1919
SN 2088-5334
PB INSIGHT - Indonesian Society for Knowledge and Human Development
K1 Hadoop; scheduler; reduce task; asynchronous; non-blocking.
AB Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.
LK http://ijaseit.insightsociety.org/index.php?option=com_content&view=article&id=9&Itemid=1&article_id=9073
DO  - 10.18517/ijaseit.10.5.9073