Poorya Mianjy – Understanding the Algorithmic Regularization due to Dropout

Abstract: Dropout is a popular algorithmic regularization technique for training deep neural networks. While it has been shown effective across a wide range of machine learning tasks — like many other popular heuristics in deep learning — dropout lacks a strong theoretical justification. In this talk, we present statistical and computational learning theoretic guarantees for dropout training in several machine learning models, including matrix sensing, deep linear networks, and two-layer ReLU networks. This talk is based primarily on the following two papers: https://arxiv.org/pdf/2003.03397.pdf, https://arxiv.org/pdf/2010.12711.pdf.

Bio: Poorya Mianjy is a Ph.D. candidate in the Department of Computer Science at the Johns Hopkins University, advised by Raman Arora. He is interested in theoretical machine learning, and in particular, the theory of deep learning.