AI Seminar: A Distribution-Aware Decision Rule for Neural Machine Translation
Join us for a talk by Bryan Eikema from the University of Amsterdam on May 11, 2022 at 1300.
Title
A Distribution-Aware Decision Rule for Neural Machine Translation
Abstract
In neural machine translation (NMT) we search for the mode of the model distribution to form predictions. The mode as well as other high probability translations found by beam search have been shown to often be inadequate in a number of ways. While this is problematic for mode-seeking search, our findings show that the model is not inherently flawed: most probability mass is put on reasonable translations. This motivates a change in decision rule that takes into account more than just the highest probability sequence. Minimum Bayes Risk is an appealing alternative that seeks for the translation with highest expected utility, akin to finding a consensus translation that is most representative of the entire model distribution under a given utility. In particular, we propose using an unbiased approximation of expected utility using ancestral samples from the model. We explore this alternative decision rule in depth and propose efficient approximations. We find that MBR scales well with more computation, appearing to not suffer from an equivalent of the beam search curse, and eventually is able to outperform beam search.
This seminar is a part of the AI Seminar Series organised by SCIENCE AI Centre. The series highlights advances and challenges in research within Machine Learning, Data Science, and AI. Like the AI Centre itself, the seminar series has a broad scope, covering both new methodological contributions, ground-breaking applications, and impacts on society.