The Quarks of Attention

Friday, March 4, 2022 1pm to 2pm

I'm Interested
Pierre Baldi
I'm Interested

About this Event

Science and Engineering Complex (SEC), LL2.221 View map Free Event

150 Western Avenue, Allston, MA 02134

https://iacs.seas.harvard.edu/event/quarks-attention
View map Free Event

RSVP HERE: https://forms.gle/smSKw4ARugNSZMwM7
Abstract: Attention plays a fundamental role in both natural and artificial intelligence systems. In deep learning, several attention-based neural network architectures have been proposed to tackle problems in natural language processing (NLP)  and beyond, including transformer architectures which currently achieve state-of-the-art performance in NLP tasks. In this presentation we will: 1) identify and classify the most fundamental building blocks (quarks) of attention, both within and beyond the standard model of deep learning; 2) identify how these building blocks are used in all current attention-based architectures, including transformers; 3) demonstrate how transformers can effectively be applied to new problems in physics, from particle physics to astronomy; and 4) present a mathematical theory of attention capacity where, paradoxically, one of the main tools in the proofs is itself an attention mechanism. 

Event Details

Event Type

Colloquia / Seminar / Lecture

Series

IACS Seminars

Academics

Applied Computation Applied Mathematics Computer Science

Audience

Alumni Faculty General Public Graduate Students Researchers / Postdocs Staff Harvard Community SEAS Community

Web Page Listing

Academic Areas Applied Computation Computer Science Applied Mathematics

Research Centers & Initiatives

Institute for Applied Computational Science (IACS)

Website

https://iacs.seas.harvard.edu/event/q...

Cost

FREE

Speaker 1

Pierre Baldi

Speaker 2

Distinguished Professor in the Department of Computer Science, School of Information and Computer Sciences

Speaker 3

Director, Institute for Genomics and Bioinformatics and Associate Director, Center for Machine Learning and Data Mining

Speaker Bio

Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Distinguished Professor in the Department of Computer Science, Director of the Institute for Genomics and Bioinformatics, and Associate Director of the Center for Machine Learning and Intelligent Systems  at the University of California Irvine. The long term focus of his research is on understanding intelligence in brains and machines. He has made contributions to the theory of deep learning, and  developed and applied deep learning methods for the natural sciences. He recently published his fifth book: Deep Learning in Science, Cambridge University Press (2021).  His honors include the 1993 Lew Allen Award at JPL, the 2010 E. R. Caianiello Prize for research in machine learning,  and election to Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.

Contact

Jackie Strom

Email

jstrom@seas.harvard.edu

See Who Is Interested

  • Sumit Sinha

1 person is interested in this event

I'm Interested