AI Seminar: Bidirectional Sequence Generation and Graph AI
Carolin Lawrence is a research scientist at NEC Labs Europe, where she works on natural language processing, graph structured data and reinforcement learning. A special interest of hers is researching ways to create explainable AI that makes a difference in praxis for real human users. Before joining NEC, she received her PhD in computational linguistics from the University of Heidelberg, where she worked in the statistical NLP group supervised by Prof. Stefan Riezler.
The talk is structured in two parts. First, we introduce a method that improves sequence generation for NLP, demonstrated on two chat bot dialogue tasks. Second, we look at various AI research projects ongoing at NEC Labs Europe, centering around using graphs to solve real world problems.
Neural sequence generation is typically performed token-by-token and left-to-right. Whenever a token is generated only previously produced tokens are taken into consideration. In contrast, for problems such as sequence classification, bidirectional attention, which takes both past and future tokens into consideration, has been shown to perform much better. We propose to make the sequence generation process bidirectional by employing special placeholder tokens. Treated as a node in a fully connected graph, a placeholder token can take past and future tokens into consideration when generating the actual output token. We verify the effectiveness of our approach experimentally on two conversational tasks where the proposed bidirectional model outperforms competitive baselines by a large margin.
Many real world problems are naturally suited to be represented as graphs. In our lab we developed two core graph AI algorithms. The first, KBlrn, can be used for knowledge base completion and we have applied it to tasks such as insurance fraud detection or multimodal knowledge graph prediction. The second, Embedding Propagation (EP), learns vector representations of nodes in a graph, with which we can tackle problems such as patient outcome prediction or personalized vaccine selection.
The seminar is free and open for everyone.
This seminar is a part of the AI Seminar Series organised by SCIENCE AI Centre. The series highlights advances and challenges in research within Machine Learning, Data Science, and AI. Like the AI Centre itself, the seminar series has a broad scope, covering both new methodological contributions, ground-breaking applications, and impacts on society.