Martin Takac, “FLECS: A Federated Learning Second-Order Framework via Compression and Sketching”
Please join us on Tuesday, April 18, 2023 at 12:00pm in CLARK HALL, Room 110 and on ZOOM for the
CIS & MINDS Seminar Series:
Guest: Martin Takac, PhD
Associate Professor
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Topic: “FLECS: A Federated Learning Second-Order Framework via Compression and Sketching”
Virtually over Zoom
Join Zoom Meeting:
https://wse.zoom.us/j/93822965644?pwd=dDNHYVZGY096QU9Dem45STBsQWQ2dz09
If you would like to meet with Martin Takac, please sign up at this link:
Martin Takac, PhD
Associate Professor
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
“FLECS: A Federated Learning Second-Order Framework via Compression and Sketching”
Abstract: Inspired by the recent work FedNL (Safaryan et al, FedNL: Making Newton-Type Methods Applicable to Federated Learning), we propose a new communication efficient second-order framework for Federated learning, namely FLECS. The proposed method reduces the high-memory requirements of FedNL by the usage of an L-SR1 type update for the Hessian approximation which is stored on the central server. A low dimensional `sketch’ of the Hessian is all that is needed by each device to generate an update, so that memory costs as well as number of Hessian-vector products for the agent are low. Biased and unbiased compressions are utilized to make communication costs also low. Convergence guarantees for FLECS are provided in both the strongly convex, and nonconvex cases, and local linear convergence is also established under strong convexity. Numerical experiments confirm the practical benefits of this new FLECS algorithm.
Biography: Martin Takac is an Associate Professor at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE. Before joining MBZUAI, he was an Associate Professor in the Department of Industrial and Systems Engineering at Lehigh University, where he has been employed since 2014. He received his B.S. (2008) and M.S. (2010) degrees in Mathematics from Comenius University, Slovakia, and Ph.D. (2014) degree in Mathematics from The University of Edinburgh, United Kingdom. His current research interests include the design and analysis of algorithms for machine learning, applications of ML, optimization, HPC. Martin received funding from various U.S. National Science Foundation programs, including through a TRIPODS Institute grant awarded to him and his collaborators at Lehigh, Northwestern, and Boston University. He served as an Associate Editor for Mathematical Programming Computation, Journal of Optimization Theory and Applications, and Optimization Methods and Software and is an area chair at machine learning conferences like ICML, NeurIPS, and AISTATS.