Hierarchical Broadcast Coding: Expediting Distributed Learning at the Wireless Edge

Dong Jun Han, Jy Yong Sohn, Jaekyun Moon

Research output: Contribution to journalArticlepeer-review


Distributed learning plays a key role in reducing the training time of modern deep neural networks with massive datasets. In this article, we consider a distributed learning problem where gradient computation is carried out over a number of computing devices at the wireless edge. We propose hierarchical broadcast coding, a provable coding-Theoretic framework to speed up distributed learning at the wireless edge. Our contributions are threefold. First, motivated by the hierarchical nature of real-world edge computing systems, we propose a layered code which mitigates the effects of not only packet losses at the wireless computing nodes but also straggling access points (APs) or small base stations. Second, by strategically allocating data partitions to nodes in the overlapping areas between cells, our technique achieves the fundamental lower bound on computational load to combat stragglers. Finally, we take advantage of the broadcast nature of wireless networks by which wireless devices in the overlapping cell coverage broadcast to more than one AP. This further reduces the overall training time in the presence of straggling APs. Experimental results on Amazon EC2 confirm the advantage of the proposed methods in speeding up learning. Our design targets any gradient descent based learning algorithms, including linear/logistic regressions and deep learning.

Original languageEnglish
Article number9285212
Pages (from-to)2266-2281
Number of pages16
JournalIEEE Transactions on Wireless Communications
Issue number4
Publication statusPublished - 2021 Apr

Bibliographical note

Funding Information:
Manuscript received March 24, 2020; revised July 7, 2020 and October 5, 2020; accepted November 19, 2020. Date of publication December 7, 2020; date of current version April 9, 2021. This work was supported by the National Research Foundation of Korea under Grant 2019R1I1A2A02061135. The associate editor coordinating the review of this article and approving it for publication was M. Kaneko. (Corresponding author: Dong-Jun Han.) The authors are with the School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea (e-mail: djhan93@kaist.ac.kr; jysohn1108@kaist.ac.kr; jmoon@kaist.edu).

Publisher Copyright:
© 2002-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Hierarchical Broadcast Coding: Expediting Distributed Learning at the Wireless Edge'. Together they form a unique fingerprint.

Cite this