A dynamic global backbone updating for communication-efficient personalised federated learning

References

• Alazab, M., Ramu, S. P., Parimala, M., Maddikunta, P. K. R., Gadekallu, T. R., & Q. V. Pham (2021). Federated learning for cybersecurity: concepts, challenges, and future directions. IEEE Transactions on Industrial Informatics, 18(5), 3501–3509. https://doi.org/10.1109/TII.2021.3119038
   Web of Science ®Google Scholar
• Amiri, M. M., Gunduz, D., Kulkarni, S. R., & Poor, H. V. (2020). Federated learning with quantized global model updates. Preprint arXiv:2006.10672.
   Google Scholar
• Arivazhagan, M. G., Aggarwal, V., Singh, A. K., & Choudhary, S. (2019). Federated learning with personalization layers. Preprint arXiv:1912.00818.
   Google Scholar
• Benesty, J., Chen, J., Huang, Y., & Cohen, I. (2009). Pearson correlation coefficient. In Noise reduction in speech processing (pp. 1–4). Springer.
   Google Scholar
• Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, B., Van Overveldt, T., Petrou, D., Ramage, D., & Roselander, J. (2019). Towards federated learning at scale: system design. In Proceedings of machine learning and systems, March 31-April 2 (pp. 374–388).
   Google Scholar
• Boopalan, P., Ramu, S. P., Pham, Q. V., Dev, K., Maddikunta, P. K. R., T. R. Gadekallu, & Huynh-The, T. (2022). Fusion of federated learning and industrial internet of things: a survey. Computer Networks, 212, 109048. https://doi.org/10.1016/j.comnet.2022.109048
   Google Scholar
• Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018). Leaf: a benchmark for federated settings. Preprint arXiv:1812.01097.
   Google Scholar
• Chen, Y., Sun, X., & Jin, Y. (2019). Communication-efficient federated deep learning with layerwise asynchronous model update and temporally weighted aggregation. IEEE Transactions on Neural Networks and Learning Systems, 31(10), 4229–4238. https://doi.org/10.1109/TNNLS.5962385
   PubMed Web of Science ®Google Scholar
• Duan, M., Liu, D., Ji, X., Liu, R., Liang, L., Chen, X., & Tan, Y. (2020). FedGroup: efficient clustered federated learning via decomposed data-driven measure. Preprint arXiv:2010.06870.
   Google Scholar
• Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018). Federated learning for mobile keyboard prediction. Preprint arXiv:1811.03604.
   Google Scholar
• Hsieh, K., Harlap, A., Vijaykumar, N., Konomis, D., Ganger, G. R., Gibbons, P. B., & Mutlu, O.. (2017). Gaia: Geo-distributed machine learning approaching LAN speeds. In 14th USENIX symposium on networked systems design and implementation (NSDI 17). ACM.
   Google Scholar
• Huang, W., Li, T., Wang, D., Du, S., Zhang, J., & Huang, T. (2022). Fairness and accuracy in horizontal federated learning. Information Sciences, 589, 170–185. https://doi.org/10.1016/j.ins.2021.12.102
   Web of Science ®Google Scholar
• Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2021). A survey on federated learning for resource-constrained iot devices. IEEE Internet of Things Journal, 9(1), 1–24. https://doi.org/10.1109/JIOT.2021.3095077
   Web of Science ®Google Scholar
• Jiang, Y., Wang, S., Valls, V., Ko, B. J., Lee, W. H., Leung, K. K., & Tassiulas, L. (2022). Model pruning enables efficient federated learning on edge devices. IEEE Transactions on Neural Networks and Learning Systems, 1–13. https://doi.org/10.1109/TNNLS.2022.3166101
   Google Scholar
• Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., D'Oliveira, R. G. L., Eichner, H., El Rouayheb, S., Evans, D., Gardner, J., Garrett, Z., Gascón, A., Ghazi, B., Gibbons, P. B., …& Zhao, S. (2021). Advances and open problems in federated learning. Foundations and Trends^TM in Machine Learning, 14(1-2), 1–210. https://doi.org/10.1561/2200000083
   Web of Science ®Google Scholar
• Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images.
   Google Scholar
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1–9. . https://doi.org/10.1145/3065386
   Google Scholar
• Kulkarni, V., Kulkarni, M., & Pant, A. (2020). Survey of personalization techniques for federated learning. In 2020 Fourth world conference on smart trends in systems, security and sustainability (WorldS4). IEEE.
   Google Scholar
• Leroy, D., Coucke, A., Lavril, T., Gisselbrecht, T., & Dureau, J.. (2019). Federated learning for keyword spotting. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 6341–6345). IEEE.
   Google Scholar
• Li, A., Sun, J., Wang, B., Duan, L., Li, S., Chen, Y., & Li, H. (2021). Lotteryfl: empower edge intelligence with personalized and communication-efficient federated learning. In 2021 IEEE/ACM symposium on edge computing (SEC). IEEE.
   Google Scholar
• Li, D., & Wang, J. (2019). Fedmd: heterogenous federated learning via model distillation. Preprint arXiv:1910.03581.
   Google Scholar
• Liang, P. P., Liu, T., Ziyin, L., Allen, N. B., Auerbach, R. P., Brent, D., Salakhutdinov, R., & Morency, L. P. (2020). Think locally, act globally: federated learning with local and global representations. Preprint arXiv:2001.01523.
   Google Scholar
• Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, W. J. (2017). Deep gradient compression: reducing the communication bandwidth for distributed training. Preprint arXiv:1712.01887.
   Google Scholar
• Liu, S., Yu, G., Yin, R., & Yuan, J. (2021). Adaptive network pruning for wireless federated learning. IEEE Wireless Communications Letters, 10(7), 1572–1576. https://doi.org/10.1109/LWC.2021.3074605
   Web of Science ®Google Scholar
• Liu, Y., Zhao, Y., Zhou, G., & Xu, K. (2021, December 8-12). FedPrune: personalized and communication-efficient federated learning on non-IID data. In International conference on neural information processing (ICONIP). Springer.
   Google Scholar
• Luo, J., Yang, J., Ye, X., Guo, X., & Zhao, W. (2021). FedSkel: efficient federated learning on heterogeneous systems with skeleton gradients update. In Proceedings of the 30th ACM international conference on information & knowledge management, November 1-5. ACM.
   Google Scholar
• McMahan, B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics (pp. 1273–1282). PMLR.
   Google Scholar
• Nguyen, D. C., Ding, M., Pham, Q. V., Pathirana, P. N., Le, L. B., Seneviratne, A., Li, J., Niyato, D., & Poor, H. V. (2021). Federated learning meets blockchain in edge computing: opportunities and challenges. IEEE Internet of Things Journal, 8(16), 12806–12825. https://doi.org/10.1109/JIOT.2021.3072611
   Web of Science ®Google Scholar
• Ramu, S. P., Boopalan, P., Pham, Q. V., Maddikunta, P. K. R., Huynh-The, T., Alazab, M., Nguyen, T. T., & Gadekallu, T. R. (2022). Federated learning enabled digital twins for smart cities: concepts, recent advances, and future directions. Sustainable Cities and Society, 79, 103663. https://doi.org/10.1016/j.scs.2021.103663
   Google Scholar
• Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2020, June). Fedpaq: a communication-efficient federated learning method with periodic averaging and quantization. In International conference on artificial intelligence and statistics, virtual, August 26-28. PMLR.
   Google Scholar
• Sattler, F., Wiedemann, S., Müller, K. R., & Samek, W. (2019). Robust and communication-efficient federated learning from non-iid data. IEEE Transactions on Neural Networks and Learning Systems, 31(9), 3400–3413. https://doi.org/10.1109/TNNLS.5962385
   PubMed Web of Science ®Google Scholar
• Shamsian, A., Navon, A., Fetaya, E., & Chechik, G. (2021, July 18-24). Personalized federated learning using hypernetworks. In International conference on machine learning, virtual. PMLR.
   Google Scholar
• Shi, S., Chu, X., Cheung, K. C., & See, S. (2019). Understanding top-k sparsification in distributed deep learning. Preprint arXiv:1911.08772.
   Google Scholar
• Shi, Y., Yu, H., & Leung, C. (2021). A survey of fairness-aware federated learning. Preprint arXiv:2111.01872.
   Google Scholar
• Shlezinger, N., Chen, M., Eldar, Y. C., Poor, H. V., & Cui, S. (2020). UVeQFed: universal vector quantization for federated learning. IEEE Transactions on Signal Processing, 69, 500–514. https://doi.org/10.1109/TSP.78
   Web of Science ®Google Scholar
• Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint arXiv:1409.1556.
   Google Scholar
• Smith, V., Chiang, C. K., Sanjabi, M., & Talwalkar, A. S. (2017). Federated multi-task learning. Advances in Neural Information Processing Systems, 30, 1–11. https://doi.org/10.5555/3294996.3295196
   Google Scholar
• Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2022). Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 1–17. https://doi.org/10.1109/TNNLS.2022.3160699
   Google Scholar
• Tao, Z., & Li, Q. (2018). eSGD: communication efficient distributed deep learning on the edge. In USENIX Workshop on hot topics in edge computing (HotEdge 18). USENIX.
   Google Scholar
• Vahidian, S., Morafah, M., & Lin, B. (2021, July 7-10). Personalized federated learning by structured and unstructured pruning under data heterogeneity. In 2021 IEEE 41st international conference on distributed computing systems workshops (ICDCSW). IEEE.
   Google Scholar
• Wang, H., Kaplan, Z., Niu, D., & Li, B. (2020, July 6-9). Optimizing federated learning on non-iid data with reinforcement learning. In IEEE INFOCOM 2020-IEEE conference on computer communications, virtual. IEEE.
   Google Scholar
• Wang, Z., Fan, X., Qi, J., Wen, C., Wang, C., & Yu, R. (2021, August 21-26). Federated learning with fair averaging. In International joint conference on artificial intelligence, virtual.
   Google Scholar
• Wu, Z., Huang, H., Zhou, Y., & Wu, C. (2022). A secure and efficient data deduplication framework for the internet of things via edge computing and blockchain. Connection Science, 34(1), 1999–2025. https://doi.org/10.1080/09540091.2022.2090514
   Web of Science ®Google Scholar
• Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. Preprint arXiv:1708.07747.
   Google Scholar
• Xin, L., Ziang, L., Yingli, Z., Wenqiang, Z., Dong, L., & Qingguo, Z. (2022). TCN enhanced novel malicious traffic detection for IoT devices. Connection Science, 34(1), 1322–1341. https://doi.org/10.1080/09540091.2022.2067124
   Web of Science ®Google Scholar
• Xu, W., Fang, W., Ding, Y., Zou, M., & Xiong, N. (2021). Accelerating federated learning for IoT in big data analytics with pruning, quantization and selective updating. IEEE Access, 9, 38457–38466. https://doi.org/10.1109/Access.6287639
   Web of Science ®Google Scholar
• Yan, X., Yin, P., Tang, Y., & Feng, S. (2022). Multi-keywords fuzzy search encryption supporting dynamic update in an intelligent edge network. Connection Science, 34(1), 511–528. https://doi.org/10.1080/09540091.2021.2023097
   Web of Science ®Google Scholar
• Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated machine learning: concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2), 1–19. https://doi.org/10.1145/3298981
   Web of Science ®Google Scholar
• Yu, F., Zhang, W., Qin, Z., Xu, Z., Wang, D., Liu, C., Tian, Z., & Chen, X. (2021, August 14-18). Fed2: feature-aligned federated learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. ACM.
   Google Scholar
• Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021). A survey on federated learning. Knowledge-Based Systems, 216, 106775. https://doi.org/10.1016/j.knosys.2021.106775
   Web of Science ®Google Scholar
• Zhang, T., Gao, L., He, C., Zhang, M., Krishnamachari, B., & Avestimehr, A. S. (2022). Federated learning for the internet of things: applications, challenges, and opportunities. IEEE Internet of Things Magazine, 5(1), 24–29. https://doi.org/10.1109/IOTM.004.2100182
   Google Scholar
• Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., & Wang, Y. (2018, September 8-14). A systematic dnn weight pruning framework using alternating direction method of multipliers. In Proceedings of the european conference on computer vision (ECCV). Springer.
   Google Scholar
• Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-iid data. Preprint arXiv:1806.00582.
   Google Scholar
• Zheng, Z., Qin, Z., Li, K., & Qiu, T. (2022). A team-based multitask data acquisition scheme under time constraints in mobile crowd sensing. Connection Science, 34(1), 1119–1145. https://doi.org/10.1080/09540091.2022.2043825
   Web of Science ®Google Scholar
• Zheng, Z., Zhou, Y., Sun, Y., Wang, Z., Liu, B., & Li, K. (2022). Applications of federated learning in smart cities: recent advances, taxonomy, and open challenges. Connection Science, 34(1), 1–28. https://doi.org/10.1080/09540091.2021.1936455
   Web of Science ®Google Scholar