Events
The Strengths and Limitations of Equivariant Neural Networks
Robin Walters, Assistant Professor, Northeastern University
Despite the success of deep learning, there remain challenges to progress. Deep models require vast datasets to train, can fail to generalize under surprisingly small changes in domain, and lack guarantees on performance. Incorporating symmetry constraints into neural networks has resulted in models called equivariant neural networks (ENN) which have helped address these challenges. I will discuss several successful applications, such as trajectory prediction, ocean currents forecasting, and robotic control. However, there are also limits to the effectiveness of current ENNs. In many applications where symmetry is only approximate or does apply across the entire input distribution, equivariance may not be the correct inductive bias to aid learning and may even hurt model performance. I will discuss recent work theoretically characterizing errors that can result from mismatched symmetry biases which can be used for model selection. I will also suggest different methods for relaxing symmetry constraints so that approximately equivariant models can still be used in these situations.
Robin Walters is an assistant professor in the Khoury College of Computer Sciences at Northeastern University, where he leads the Geometric Learning Lab. Robin’s research seeks to develop a fundamental understanding of the role symmetry plays in deep learning and to exploit this to improve the generalization and data efficiency of deep learning methods. This includes designing equivariant neural networks, symmetry discovery methods, and creating a theory of symmetry for model parameters. Previously, Robin studied representation theory and algebraic geometry at UChicago and Harvard and worked on Toy Story 3D at Pixar.
