Events

IACS Seminars

The Quarks of Attention

Pierre Baldi

Distinguished Professor in the Department of Computer Science, School of Information and Computer Sciences

Director, Institute for Genomics and Bioinformatics and Associate Director, Center for Machine Learning and Data Mining

Mar 4, 2022

RSVP HERE: https://forms.gle/smSKw4ARugNSZMwM7
Abstract: Attention plays a fundamental role in both natural and artificial intelligence systems. In deep learning, several attention-based neural network architectures have been proposed to tackle problems in natural language processing (NLP)  and beyond, including transformer architectures which currently achieve state-of-the-art performance in NLP tasks. In this presentation we will: 1) identify and classify the most fundamental building blocks (quarks) of attention, both within and beyond the standard model of deep learning; 2) identify how these building blocks are used in all current attention-based architectures, including transformers; 3) demonstrate how transformers can effectively be applied to new problems in physics, from particle physics to astronomy; and 4) present a mathematical theory of attention capacity where, paradoxically, one of the main tools in the proofs is itself an attention mechanism. 

Speaker Bio

Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Distinguished Professor in the Department of Computer Science, Director of the Institute for Genomics and Bioinformatics, and Associate Director of the Center for Machine Learning and Intelligent Systems  at the University of California Irvine. The long term focus of his research is on understanding intelligence in brains and machines. He has made contributions to the theory of deep learning, and  developed and applied deep learning methods for the natural sciences. He recently published his fifth book: Deep Learning in Science, Cambridge University Press (2021).  His honors include the 1993 Lew Allen Award at JPL, the 2010 E. R. Caianiello Prize for research in machine learning,  and election to Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.

Contact
Contact:

Jackie Strom

Location