Distributed dual coordinate ascent in general tree networks and communication network effect on synchronous machine learning

Due to the big size of data and limited data storage volume of a single computer or a single server, data are often stored in a distributed manner. Thus, performing large-scale machine learning operations with the distributed datasets through communication networks is often required. In this paper, we study the convergence rate of the distributed dual coordinate ascent for distributed machine learning problems in a general tree-structured network. Since a tree network model can be understood as the generalization of a star network, our algorithm can be thought of as the generalization of the distributed dual coordinate ascent in a star network. We provide the convergence rate of the distributed dual coordinate ascent over a general tree network in a recursive manner and analyze the network effect on the convergence rate. Secondly, by considering network communication delays, we optimize the distributed dual coordinate ascent algorithm to maximize its convergence speed. From our analytical result, we can choose the optimal number of local iterations depending on the communication delay severity to achieve the fastest convergence speed. In numerical experiments, we consider machine learning scenarios over communication networks, where local workers cannot directly reach to a central node due to constraints in communication, and demonstrate that the usability of our distributed dual coordinate ascent algorithm in tree networks.

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Files

Metadata

Work Title Distributed dual coordinate ascent in general tree networks and communication network effect on synchronous machine learning
Access
Open Access
Creators
  1. Myung Cho
  2. Lifeng Lai
  3. Weiyu Xu
License In Copyright (Rights Reserved)
Work Type Article
Publisher
  1. Institute of Electrical and Electronics Engineers (IEEE)
Publication Date July 2021
Publisher Identifier (DOI)
  1. 10.1109/jsac.2021.3078495
Source
  1. IEEE Journal on Selected Areas in Communications
Deposited June 17, 2022

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created
  • Added main_JSAC_R2_Final_double-1.pdf
  • Added Creator Myung Cho
  • Added Creator Lifeng Lai
  • Added Creator Weiyu Xu
  • Published
  • Updated