Orhan Firat Seminar

Orhan Firat - Research Scientist at Google Research

Abstract: Multitask learning aims at leveraging the information contained in the training signals of various related tasks to improve the learning for one task. Recent years have witnessed a significant interest in building massively multitask neural networks for Natural Language Processing, where hundreds of tasks (languages) are trained together as the model sizes grow almost exponentially in tandem. In this talk, we will share our journey and insights building Machine Translation models that can translate between 1000 languages when the underlying neural network surpasses 1 trillion parameters. Being one of the largest stress tests for multitask learning, we will be sharing several research (and even some development) challenges that we have faced; multi-task learning with thousands of tasks, learning under heavy data imbalance, building very large models while sharing sub-spaces and their optimization difficulties.

Brief Bio: Orhan Firat (PhD), is a Research Scientist at Google Research working on the areas of sequence modeling, multilingual models, multi-task learning and scaling neural networks.