Stochastic optimization algorithms like gradient descent are a fundamental tool in modern machine learning. However, our understanding of why some algorithms are better than others is woefully incomplete - in many cases the empirical evidence seems to directly contradict standard theoretical guidelines! In this talk, I will highlight some fundamental gaps in our knowledge, and show how we can partially address these gaps using tools from online learning. Online learning is a seemingly unrelated field that studies sequential decision making in a potentially adversarial environment. Perhaps surprisingly, online learning is connected to many areas of machine learning, including stochastic optimization. However, these classical results do not shed too much light on modern empirical techniques. By building new and improved reductions from stochastic optimization to online learning, we will gain a better understanding of current algorithms used in practice, and even find directions for future algorithm design.

Location: SEC 1.307 and Zoom (Password: 599436)