AI Seminar: Beyond supervised learning: generalization, few-shot learning, and robustness

AI Seminar

At this seminar, Jonathan Berant will describe the recent work that goes beyond the traditional paradigm of natural language understanding. The seminar is relevant to people who are interested in machine learning and natural language processing.

Title

AI seminar: Beyond supervised learning: generalization, few-shot learning, and robustness

Abstract

Pre-trained language models combined with fine-tuning on labeled data have led to impressive results throughout natural language understanding. In this talk, I will describe some recent work that goes beyond this traditional paradigm. First, I will present a class of models with an inductive bias towards tree structures and show that such models exhibit much better compositional generalization, that is, they generalize better to new compositions. Second, I will describe Splinter, a pre-training technique that emulates question answering during pre-training and leads to large gains in the few-shot setup. For example, Splinter achieves 73 F1 on SQuAD with as few as 128 examples. Time permitting, I will also describe some recent work on improving the robustness of models through adversarial training, where we propose a new discrete attack and show it can substantially improve robustness through online augmentation.

SpeakerPortrait of Jonathan Berant

Jonathan Berant is an associate professor at the School of Computer Science at Tel Aviv University and a research scientist at The Allen Institute for AI.  Jonathan earned a Ph.D. in Computer Science at Tel-Aviv University, under the supervision of Prof. Ido Dagan. Jonathan was a post-doctoral fellow at Stanford University, working with Prof. Christopher Manning and Prof. Percy Liang, and subsequently a post-doctoral fellow at Google Research, Mountain View. Jonathan Received several awards and fellowships including The Rothschild fellowship, The ACL 2011 best student paper award, EMNLP 2014 best paper award, and NAACL 2019 best resource paper award, as well as several honorable mentions. Jonathan is currently an ERC grantee.