Federated Learning of Large Model at the Edge via Principal Sub-Model Training

Yue Niu, Saurav Prakash, Souvik Kundu, Sunwoo Lee, Salman Avestimehr

NeurIPS Workshop, 2022

arXiv

This work proposes PriSM, a principal sub-model training methodology for cross-device Federated Learning that assigns each resource-constrained client a small, importance-sampled low-rank sub-model derived via principal kernel analysis, letting clients collaboratively train a full large model without any client training it fully or sharing intermediate information, while together achieving near-full coverage of the model’s principal kernels.

BibTeX

@article{niu2022federated,
  title={Federated learning of large models at the edge via principal sub-model training},
  author={Niu, Yue and Prakash, Saurav and Kundu, Souvik and Lee, Sunwoo and Avestimehr, Salman},
  journal={arXiv preprint arXiv:2208.13141},
  year={2022}
}