Improving Scalability of Parallel CNN Training by Adaptively Adjusting Parameter Update Frequency

Sunwoo Lee, Qiao Kang, Reda Al-Bahrani, Ankit Agrawal, Alok Choudhary, Wei-keng Liao

Journal of Parallel and Distributed Computing (JPDC), 2022

This study proposes a partial-model update frequency adjustment method which is designed to reduce the communication cost in centralized parallel training.

BibTeX

@article{lee2022improving,
  title={Improving scalability of parallel CNN training by adaptively adjusting parameter update frequency},
  author={Lee, Sunwoo and Kang, Qiao and Al-Bahrani, Reda and Agrawal, Ankit and Choudhary, Alok and Liao, Wei-Keng},
  journal={Journal of Parallel and Distributed Computing},
  volume={159},
  pages={10--23},
  year={2022},
  publisher={Elsevier}
}