Events
The Quarks of Attention
Pierre Baldi
Distinguished Professor in the Department of Computer Science, School of Information and Computer Sciences
Director, Institute for Genomics and Bioinformatics and Associate Director, Center for Machine Learning and Data Mining
RSVP HERE: https://forms.gle/smSKw4ARugNSZMwM7
Abstract: Attention plays a fundamental role in both natural and artificial intelligence systems. In deep learning, several attention-based neural network architectures have been proposed to tackle problems in natural language processing (NLP) and beyond, including transformer architectures which currently achieve state-of-the-art performance in NLP tasks. In this presentation we will: 1) identify and classify the most fundamental building blocks (quarks) of attention, both within and beyond the standard model of deep learning; 2) identify how these building blocks are used in all current attention-based architectures, including transformers; 3) demonstrate how transformers can effectively be applied to new problems in physics, from particle physics to astronomy; and 4) present a mathematical theory of attention capacity where, paradoxically, one of the main tools in the proofs is itself an attention mechanism.
Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Distinguished Professor in the Department of Computer Science, Director of the Institute for Genomics and Bioinformatics, and Associate Director of the Center for Machine Learning and Intelligent Systems at the University of California Irvine. The long term focus of his research is on understanding intelligence in brains and machines. He has made contributions to the theory of deep learning, and developed and applied deep learning methods for the natural sciences. He recently published his fifth book: Deep Learning in Science, Cambridge University Press (2021). His honors include the 1993 Lew Allen Award at JPL, the 2010 E. R. Caianiello Prize for research in machine learning, and election to Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.
People Interested (1)
 |