Skip to main content
Upcoming Events:

Seminar: Shahab Asoodeh

Date & Time:
   Add All to Calendar

Information Theory for Responsible Machine Learning



In this talk, I first illustrate some of the challenges that modern machine learning algorithms face, such as privacy and discrimination. Then, in the first part of the talk, I focus on privacy and describe the differential privacy framework—the de facto standard for privacy-preserving data analytics— and show that it can be equivalently expressed in terms of a certain divergence between two probability distributions, known as “hockey-stick divergence”. This equivalent perspective enables us to exploit information theory for better understanding privacy-accuracy trade-offs in iterative machine learning algorithms. As an example, I consider the stochastic gradient descent algorithm and show that a simple information-theoretic manipulation allows us to run the algorithm for 100 more iterations for training deep learning models with the same privacy guarantee than the state-of-the-art. In the second part, I discuss the discriminatory behaviors of some machine learning algorithms and give some mathematical definitions of fairness. Then, I argue that the corresponding optimal fairness-accuracy trade-offs can be cast in terms of an optimization problem that is remarkably similar to a classical problem in information theory introduced in 70’s, known as “information projection”. Under some regularity conditions, this optimization problem can be solved, and its solution leads to a blueprint for bias-correcting machine learning models post-deployment. Time permitting, I also discuss “privacy watchdog”, an information-theoretic framework for the real-time monitoring of potential privacy breaches before tweeting or posting an image on Facebook.


Shahab Asoodeh is a postdoctoral scholar in the School of Engineering and Applied Sciences at Harvard University. Prior to that, he was a postdoctoral fellow in the Knowledge Lab at The University of Chicago. He received his PhD in Applied Mathematics from Queen’s University in 2017 and his M.Sc. in Electrical Engineering jointly from ETH Zürich and TU Delft in 2012. His research interests include information-theoretic approaches in machine learning, differential privacy, algorithmic fairness, and discrete differential geometry.