CROSS-LINGUAL VOICE CONVERSION

Abstract

The thesis focuses on the cross-lingual voice conversion (XVC) that aims to modify a source speaker identity towards a target while preserving the source linguistic content. Firstly, we study the use of bilingual information in XVC including the bilingual linguistic representation and average modeling approach, where the model is trained with multi-speaker database in two languages. Secondly, we propose Multi-Task WaveRNN with an integrated architecture for XVC. It maps the input linguistic information directly to the speech waveform, and the network is developed in two steps with multi-task learning. Thirdly, the language agnostic speaker embedding is proposed using an encoder-decoder architecture that disentangles the language information from speaker embeddings via multi-task learning. The proposed speaker embedding is verified in both XVC and text-to-speech synthesis tasks. Lastly, we investigate the problem of non-native accent in current XVC techniques and propose to incorporate an additional linguistic consistency loss into the XVC network to make the converted speech sound native.