Rebekka Burkholz, “Pruning deep neural networks for lottery tickets”
“Pruning deep neural networks for lottery tickets”
Rebekka Burkholz, PhD
Helmholtz Center for Information Security (CISPA)
Germany
Abstract: The strong lottery ticket hypothesis (SLTH) states that a sufficiently large, randomly initialized neural network contains subnetworks that can approximate neural networks of smaller size with high probability. This hypothesis holds the promise that pruning random networks could offer a computationally efficient alternative to deep learning with stochastic gradient descent and has sparked the rapid development of new pruning algorithms. By proving the SLTH for realistic architectures, we will identify conditions when these algorithms have a chance to succeed. Inspired by our proofs, we will further create a test environment that highlights current limitations of state-of-the-art algorithms in finding extremely sparse tickets. We will conclude with some opportunities for future progress.
Biography: Rebekka Burkholz holds a tenure-track faculty at the Helmholtz Center for Information Security (CISPA) in Germany. The big goal of her research is to gain a theoretical understanding of deep learning from a complex network perspective and to improve contemporary algorithms based on these insights.
She recently was a PostDoc at the Biostatistics Department of the Harvard T.H. Chan School of Public Health, where she worked with John Quackenbush. Before that, she enjoyed postdoctoral research at ETH Zurich at the Institute for Machine Learning and earned her PhD with Frank Schweitzer at the Chair of Systems Design. Her Phd thesis on systemic risk at the ETH Risk Center won the Zurich Dissertation Prize and her work on international maize trade received the CSF Best Contribution Award. She studied Mathematics and Physics at TU Darmstadt.
Join Zoom Meeting:
Join Zoom Meeting
https://wse.zoom.us/j/99304114570
Meeting ID: 993 0411 4570
One tap mobile
+13017158592,,99304114570# US (Washington DC) 13126266799,,99304114570#
+US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
Meeting ID: 993 0411 4570
Find your local number: https://wse.zoom.us/u/acPT2svkU3
Join by SIP
Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 993 0411 4570