CROSS-LINGUAL LANGUAGE MODELING: METHODS AND APPLICATIONS

Abstract

Cross-lingual learning aims to connect the numerous languages in the world by bringing monolingual systems into a multi-lingual space through which the system can realize cross-lingual transfer in tasks like zero-shot POS tagging, retrieval or classification. It also enables applications like the modeling of code-switching languages. These applications present different challenges, such as the data is often small, and in terms of the code-switching domain, the data is sparse. A more systematic challenge is the performance degradation related to the language distance and the difference in language structures and domains. This thesis seeks to address some of these questions. Firstly, we propose an information-theoretic framework for understanding the working of unsupervised cross-lingual learning. Secondly, we delve into the linguistically motivated data augmentation method. Next, we propose a novel neural back-off scheme in the language model. Lastly, we validate our methods in downstream tasks such as speech recognition and synthesis.