Maria De-Arteaga
Title: Mind the gap: From predictions to ML-informed decisions
Abstract: Machine learning (ML) is increasingly being used to support decision-making in critical settings, where predictions have potentially grave implications over human lives. In this talk, I will discuss the gap that exists between ML predictions and ML-informed decisions. The first part of the talk will highlight the role of humans-in-the-loop through a study of the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We focus on the question: Are humans capable of identifying cases in which the machine is wrong, and of overriding those recommendations? The second part of the talk will focus on the gap between the observed outcome that the algorithm optimizes for and the construct of interest to experts. We propose influence functions based methodology to reduce this gap by extracting knowledge from experts’ historical decisions. In the context of child maltreatment hotline screenings, we find that (1) there are high-risk cases whose risk is considered by the experts but not wholly captured in the target labels used to train a deployed model, and (2) the proposed approach improves recall for these cases.
Bio: Maria De-Arteaga is an Assistant Professor at the Information, Risk and Operation Management (IROM) Department at the University of Texas at Austin, where she is also a core faculty member in the Machine Learning Laboratory. She received a joint PhD in Machine Learning and Public Policy from Carnegie Mellon University. Her research focuses on the risks and opportunities of using machine learning to support experts’ decisions in high-stakes settings. Her work has been awarded the Best Thematic Paper Award at NAACL’19, the Innovation Award on Data Science at Data for Policy’16, and has been featured by UN Women and Global Pulse in their report Gender Equality and Big Data: Making Gender Data Visible. She is a recipient of a 2020 Google Award for Inclusion Research, a 2018 Microsoft Research Dissertation Grant, and was named an EECS 2019 Rising Star.