Curriculum Design for Code-switching: Experiments with Language Identification and Language Modeling with Deep Neural Networks

Curriculum learning strategies are known to improve the accuracy, robustness and convergence rate for various language learning tasks using deep architectures (Bengio et al., 2009). In this work, we design and experiment with several training curricula for two tasks – word-level language detection and language modeling – for code-switched text data. Our study shows that irrespective of the task or the underlying DNN architecture, the best curriculum for training the code-switched models is to first train a network with monolingual training instances, where each mini-batch has instances from both languages, and then train the resulting network on code-switched data.