Distributed learning on correlated data with communication constraints

Carlee Joe-Wong
Associate Professor, Electrical and Computer Engineering
Carnegie Mellon University

Abstract: As more and more devices collect increasing amounts of data about their surrounding environments and their users’ behavior, there is a growing need for machine learning frameworks that can learn from data that has been collected by multiple different devices, e.g., multiple Internet-of-Things sensors or servers in different countries. Many such frameworks fall under the federated learning umbrella, which aims to train a machine learning model while keeping data at the devices where it is collected. Most federated learning frameworks, however, do not account for correlations between the data collected at different clients (i.e., devices). Such correlations require clients to exchange information in order to train accurate models, which may impose significant communication costs or violate communication constraints. In this talk, I will present two recent works motivated by applications in which such correlations exist. First, I will discuss federated learning for node classification tasks in graphs, motivated by social networking and recommendation applications. In such applications, nodes in a large graph are partitioned across clients, e.g., users in a social network partitioned across countries. Privacy constraints prevent sending raw data between clients, yet the presence of cross-client edges in the graph (e.g., if users in different countries are friends with each other) means that some information exchange between clients is needed to learn an accurate node classification model. Second, I will discuss environmental sensing on Internet-of-Things devices. In this application, multiple sensors may wish to conduct data fusion across their sensor readings, e.g., to track the path of a wildfire. However, power or communication constraints can prevent them from freely exchanging data for the fusion. For both applications, I will present optimized strategies for exchanging information among clients that allow an operator to navigate and optimize the tradeoffs between exchanging more information and training a more accurate model.

Bio: Carlee Joe-Wong is the Robert E. Doherty Associate Professor of Electrical and Computer Engineering at Carnegie Mellon University. She received her A.B. degree (magna cum laude) in Mathematics, and M.A. and Ph.D. degrees in Applied and Computational Mathematics, from Princeton University in 2011, 2013, and 2016, respectively. Her research interests lie in optimizing various types of networked systems, including applications of machine learning and pricing to cloud computing, mobile/wireless networks, and transportation networks. From 2013 to 2014, she was the Director of Advanced Research at DataMi, a startup she co-founded from her research on mobile data pricing. She received the NSF CAREER award in 2018 and the ARO Young Investigator award in 2019.