Spring 2020
Prof. Justin Romberg (Georgia Tech)
Date: Friday, January 31, 2020
Time: 11:00am – 12:00pm
Location: EER 1.516
Title: Distributed Stochastic Approximation: Reinforcement Learning and Optimization with Communication Constraints
Abstract:
We will discuss two problems that have different application spaces but share a common mathematical core. These problems combine stochastic approximation, an iterative method for finding the fixed point of a function from noisy observations, and consensus, a general averaging technique for multiple agents to cooperatively solve a distributed problem.
In the first part of the talk, we will discuss the policy evaluation problem in multi-agent reinforcement learning. In this problem, a set of agents operate in a common environment under a fixed control policy, working together to discover the value (accumulative reward) associated with each environmental state. We give a finite-time analysis on the performance of the well-known “TD-lambda” algorithm that depends on the connectivity of the agents and the intrinsic properties of the Markov decision process driving the agents decisions.
In the second part, we discuss distributed optimization problems over a network when the communication between the nodes is constrained, and so information that is exchanged between the nodes must be quantized. We propose a modified consensus-based gradient method that communicates using random (dithered) quantization. We derive an upper bounds on the rate of convergence of this method as a function of the bandwidth available between the nodes and the underlying network topology, and derive rates for both convex and strongly convex objective functions.
Bio: Dr. Justin Romberg is a Professor in the School of Electrical and Computer Engineering at the Georgia Institute of Technology. Dr. Romberg received the B.S.E.E. (1997), M.S. (1999) and Ph.D. (2004) degrees from Rice University in Houston, Texas. From Fall 2003 until Fall 2006, he was a Postdoctoral Scholar in Applied and Computational Mathematics at the California Institute of Technology. He spent the Summer of 2000 as a researcher at Xerox PARC, the Fall of 2003 as a visitor at the Laboratoire Jacques-Louis Lions in Paris, and the Fall of 2004 as a Fellow at UCLA’s Institute for Pure and Applied Mathematics. In the Fall of 2006, he joined the Georgia Tech ECE faculty. In 2008 he received an ONR Young Investigator Award, in 2009 he received a PECASE award and a Packard Fellowship, and in 2010 he was named a Rice University Outstanding Young Engineering Alumnus. In 2006-2007 he was a consultant for the TV show “Numb3rs”. He was an Associate Editor for the IEEE Transactions on Information Theory from 2008-2011, for the SIAM Journal on Imaging Science from 2013-2018, and the SIAM Journal on the Mathematics of Data Science from 2018-present. In 2018, he was named an IEEE Fellow.
Prof. Volkan Cevher (EPFL)
Date: Monday, February 3, 2020
Time: 3:00 pm – 4:00 pm
Location: EER 3.646
Title: Robust Reinforcement Learning with Langevin Dynamics
Abstract:
In this talk, I will talk about principled ways of solving a classical reinforcement learning (RL) problem and introduce its robust variant.
In particular, we rethink the exploration-exploitation trade-off in RL as an instance of a distribution sampling problem in infinite dimensions. Using the powerful Stochastic Gradient Langevin Dynamics (SGLD), we propose a new RL algorithm, which results in a sampling variant of the Twin Delayed Deep Deterministic Policy Gradient (TD3) method. Our algorithm consistently outperforms existing exploration strategies for TD3 based on heuristic noise injection strategies in several MuJoCo environments.
The sampling perspective enables us to introduce an action-robust variant of RL objective, which is as a particular case of a zero-sum two-player Markov game. In this setting, at each step of the game, both players simultaneously choose an action. The reward each player gets after one step depends on the state and the convex combination of the actions of both players. Based on our earlier work (SGLD for min-max/GAN problem), we propose a new robust RL algorithm with convergence guarantee and provide numerical evidence of the new algorithm. Finally, I will also discuss future directions on the application of the framework to self-play in games.
Bio: Volkan Cevher received the B.Sc. (valedictorian) in electrical engineering from Bilkent University in Ankara, Turkey, in 1999 and the Ph.D. in electrical and computer engineering from the Georgia Institute of Technology in Atlanta, GA in 2005. He was a Research Scientist with the University of Maryland, College Park from 2006-2007and also with Rice University in Houston, TX, from 2008-2009. Currently, he is an Associate Professor at the Swiss Federal Institute of Technology Lausanne and a Faculty Fellow in the Electrical and Computer Engineering Department at Rice University. His research interests include signal processing theory, machine learning, convex optimization, and information theory. Dr. Cevher was the recipient of the IEEE Signal Processing Society Best Paper Award in 2016, a Best Paper Award at CAMSAP in 2015, a Best Paper Award at SPARS in 2009, and an ERC CG in 2016 as well as an ERC StG in 2011.
Prof. Ameet Talwalkar (Carnegie Mellon University)
Date: Friday, February 21, 2020
Time: 11:00 am – 12:00 pm
Location: EER 1.516
Title: Toward the Jet Age of Machine Learning
Abstract:
Machine learning today bears resemblance to the field of aviation soon after the Wright Brothers’ pioneering flights in the early 1900s. It took half a century of aeronautical engineering advances for the ‘Jet Age’ (i.e., commercial aviation) to become a reality. Similarly, machine learning (ML) is currently experiencing a renaissance, yet fundamental barriers must be overcome to fully unlock the potential of ML-powered technology. In this talk, I describe our work to help democratize ML by tackling barriers related to scalability, privacy, and safety. In the context of scalability and privacy, I discuss theoretically principled, privacy-preserving approaches to federated learning (i.e., learning over massive networks of edge devices) that rely on novel connections to gradient-based meta-learning. In the context of safety, we reduce the gap between model transparency and model accuracy via a novel model family of interpretable random forests that also serves as a state-of-the-art black-box explanation system.
Bio: Ameet Talwalkar is an assistant professor in the Machine Learning Department at CMU, and also co-founder and Chief Scientist at Determined AI. His interests are in the field of statistical machine learning. His current work is motivated by the goal of democratizing machine learning, with a focus on topics related to scalability, automation, fairness, and interpretability of learning algorithms and systems. He led the initial development of the MLlib project in Apache Spark, is a co-author of the textbook ‘Foundations of Machine Learning’ (MIT Press), and created an award-winning edX MOOC on distributed machine learning. He also helped to create the MLSys conference, serving as the inaugural Program Chair in 2018, General Chair in 2019, and currently as President of the MLSys Board.
Prof. Rahul Jain (USC)
Date: April 24, 2020
Time: 11:00am – 12:00pm
Location: EER 1.516
Title: TBD
Abstract: TBD
Bio: Rahul Jain is the Kenneth C. Dahlberg Early Career Chair in the Electrical Engineering department and an Associate Professor since February 2013 at the University of Southern California, Los Angeles, CA. Prior to joining USC in August 2008, he was at the Mathematical Sciences Division of the IBM T.J. Watson Research Center, Yorktown Heights, NY. He received his Ph.D. in EECS in December 2004, and an M.A. in Statistics, both from the University of California, Berkeley. He also received an M.S. in ECE from Rice University where he was a recipient of the Texas Instruments Fellowship. He received a B.Tech in EE from the Indian Institute of Technology, Kanpur, where he won the TCS Best B.Tech Project Award. He has received numerous awards including a James H. Zumberge Faculty Research and Innovation Award in 2009, an IBM Faculty Award, the NSF CAREER Award in 2010 and the ONR Young Investigator Award in 2012. He is a US Fulbright Specialist Scholar for 2017-2020, and was a Fulbright Visiting Professor at the Indian Institute of Science, Bangalore in 2018. He has diverse research interests with current focus on Stochastic Control, Online, Statistical and Reinforcement Learning. He has also worked on Queueing Systems, Power System Economics, Network Economics and Game Theory. He is currently an Associate Editor for IEEE Trans. on Automatic Control, IEEE Trans. on Control of Networked Systems, IEEE T. on Network Science & Engineering and serves on the Conference Editorial Board of IEEE Control Systems Society. In the past, he has been an AE for IEEE. T. on Mobile Computing (2013-16) and was the lead Guest editor of a special issue in the IEEE J. Selected Areas in Communications, Associate editor for (2014-present). He has served as the TPC Co-Chair of GameNets 2011, and as a TPC member of many conferences.
Prof. Rayadurgam Srikant (UIUC)
Date: May 1, 2020
Time: 11:00am – 12:00pm
Location: EER 1.516
Title: TBD
Abstract: TBD
Bio: R. Srikant is the Fredric G. and Elizabeth H. Nearing Endowed Professor of Electrical and Computer Engineering and the Coordinated Science Lab at the University of Illinois at Urbana-Champaign. His research interests are in the areas of communication networks, cloud computing, applied probability, and machine learning. He received the 2015 IEEE INFOCOM Achievement Award, and has also received several Best Paper awards, including the 2015 INFOCOM Best Paper Award and the 2017 Applied Probability Society Best Publication Award. He was the Editor-in-Chief of the IEEE/ACM Transactions on Networking from 2013-2017.
Prof. Richard Baraniuk (Rice University)
Date: TBD
Time: 11:00am – 12:00pm
Location: EER 1.516
Title: TBD
Abstract: TBD
Bio: Richard G. Baraniuk is the Victor E. Cameron Professor of Electrical and Computer Engineering at Rice University. His research interests lie in new theory, algorithms, and hardware for sensing, signal processing, and machine learning. He is a Fellow of the IEEE, AAAS, and the National Academy of Inventors and has received the DOD Vannevar Bush Faculty Fellowship (NSSEFF), national young investigator awards from the US NSF and ONR, the Rosenbaum Fellowship from the Isaac Newton Institute of Cambridge University, the ECE Young Alumni Achievement Award from the University of Illinois, the Wavelet Pioneer and Compressive Sampling Pioneer Awards from SPIE, the IEEE Signal Processing Society Best Paper Award, and the IEEE Signal Processing Society Technical Achievement Award. His work on the Rice single-pixel compressive camera has been widely reported in the popular press and was selected by MIT Technology Review as a TR10 Top 10 Emerging Technology. For his teaching and education projects, including Connexions (cnx.org) and OpenStax (openstax.org), he has received the C. Holmes MacDonald National Outstanding Teaching Award from Eta Kappa Nu, the Tech Museum of Innovation Laureate Award, the Internet Pioneer Award from Harvard University, the World Technology Award for Education, the IEEE-SPS Education Award, the WISE Education Award, and the IEEE James H. Mulligan, Jr. Medal for Education.
Prof. Hamed Hassani (UPenn)
Date: TBD
Time: 11:00am – 12:00pm
Location: EER 1.516
Title: TBD
Abstract: TBD
Bio: Hamed Hassani is currently an assistant professor of Electrical and Systems Engineering department at the University of Pennsylvania. Prior to that, he was a research fellow at Simons Institute for the Theory of Computing (UC Berkeley) affiliated with the program of Foundations of Machine Learning, and a post-doctoral researcher in the Institute of Machine Learning at ETH Zurich. He received a Ph.D. degree in Computer and Communication Sciences from EPFL, Lausanne. He is the recipient of the 2014 IEEE Information Theory Society Thomas M. Cover Dissertation Award, 2015 IEEE Jack Keil Wolf ISIT Student Paper Award, 2017 Simons-Berkeley Fellowship, and 2018 NSF-CRII Research Initiative Award.
