Greg Ongie, “A function space view of infinite-width neural networks”
Greg Ongie, PhD
Assistant Professor
Mathematical and Statistical Sciences
Marquette University
Abstract: It is well-known that nearly any function can be approximated arbitrarily-well by a neural network with non-linear activations. However, one cannot guarantee that the weights in the neural network remain bounded in norm as the approximation error goes to zero, which is an important consideration when practically training neural networks. This raises the question: What functions are well-approximated by neural networks with bounded norm weights? In this talk, I will give a partial answer to this question, by giving a precise characterization of the space of functions that can be approximated arbitrarily well by a two-layer neural network with ReLU activations having an unbounded number of units (“infinite-width”) but whose weights remain bounded in norm. Surprisingly, the characterization relates to the Radon transform as used in computational imaging, and I will show how Radon transform analysis yields new insights about function approximation with two-layer and three-layer neural networks.
Biography: Dr. Greg Ongie is an Assistant Professor of Mathematical and Statistical Sciences at Marquette University in Milwaukee, WI. Prior to this, he held postdoctoral positions at University of Chicago and University of Michigan. He received the PhD in Applied Mathematics from University of Iowa in 2016. His research interests include computational imaging, machine learning, and the mathematical foundations of deep learning.
In person – Kavli Room, Clark 316
Or over Zoom (information below)
Join Zoom Meeting
https://wse.zoom.us/j/99304114570
Meeting ID: 993 0411 4570
One tap mobile
+13017158592,,99304114570# US (Washington DC) 13126266799,,99304114570#
+US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
Meeting ID: 993 0411 4570
Find your local number: https://wse.zoom.us/u/acPT2svkU3
Join by SIP
Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 993 0411 4570