AI Seminar: Sequence Generation and Graph AI

Abstract

The talk is structured in two parts. First, we introduce a method that improves sequence generation for NLP, demonstrated on two chatbot dialogue tasks. Second, we look at various AI research projects ongoing at NEC Labs Europe, centering around using graphs to solve real-world problems.

Part 1:
Neural sequence generation is typically performed token-by-token and left-to-right. Whenever a token is generated only previously produced tokens are taken into consideration. In contrast, for problems such as sequence classification, bidirectional attention, which takes both past and future tokens into consideration, has been shown to perform much better. We propose to make the sequence generation process bidirectional by employing special placeholder tokens. Treated as a node in a fully connected graph, a placeholder token can take past and future tokens into consideration when generating the actual output token. We verify the effectiveness of our approach experimentally on two conversational tasks where the proposed bidirectional model outperforms competitive baselines by a large margin.

Part 2:
Many real-world problems are naturally suited to be represented as graphs. In our lab, we developed two core graph AI algorithms. The first, KBlrn, can be used for knowledge base completion and we have applied it to tasks such as insurance fraud detection or multimodal knowledge graph prediction. The second, Embedding Propagation (EP), learns vector representations of nodes in a graph, with which we can tackle problems such as patient outcome prediction or personalized vaccine selection.

Speaker bio

Carolin Lawrence (https://carolinlawrence.github.io/) is a research scientist at NEC Labs Europe (https://uk.nec.com/en_GB/emea/about/neclab_eu), where she works on natural language processing, graph-structured data and reinforcement learning. A special interest of hers is researching ways to create explainable AI that makes a difference in praxis for real human users. Before joining NEC, she received her PhD in computational linguistics from the University of Heidelberg, where she worked in the statistical NLP group supervised by Prof. Stefan Riezler.