How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
February 2023
Project Z-Code
Publications
Publications by Year
-
-
Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production
Hany Hassan Awadalla, YOUNG JIN KIM
December 2022
Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPU
Hossam Amer, Young Jin Kim, Mohamed Afify, Hitokazu Matsushita, Hany Hassan Awadalla
AMTA | August 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
Muhammad ElNokrashy, Amr Hendy, Mohamed Maher, Mohamed Afify, Hany Hassan Awadalla
AMTA | August 2022
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Rui Liu, Young Jin Kim, Alexandre Muzio, Hany Hassan Awadalla
ICML 2022 | July 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi, Shufang Xie, Tao Qin, Hany Hassan Awadalla
NAACL 2022 (The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies) | July 2022
Taming Sparsely Activated Transformer with Stochastic Experts
Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
ICLR 2022 | April 2022
-
Improving Multilingual Translation by Representation and Gradient Regularization
Yilin Yang, Akiko Eriguchi, Alexandre Muzio, Prasad Tadepalli, Stefan Lee, Hany Hassan Awadalla
EMNLP 2021 (The 2021 Conference on Empirical Methods in Natural Language Processing) | November 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task.
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
WMT | November 2021
Discovering Representation Sprachbund For Multilingual Pre-Training
Yimin Fan, Yaobo Liang, Alexandre Muzio, Hany Hassan Awadalla, Houqiang Li, Ming Zhou, Nan Duan
EMNLP 2021 | October 2021
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio, Felipe Cruz Salinas, Liyang Lu, Amr Hendy, Samyam Rajbhandari, Yuxiong He, Hany Hassan Awadalla
September 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
arXiv: Computation and Language | June 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
June 2021
-
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders.
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei
December 2020
Multi-task Learning for Multilingual Neural Machine Translation
Yiren Wang, Hany Hassan Awadalla
EMNLP | December 2020
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding
Young Jin Kim, Hany Hassan Awadalla
SustaiNLP – EMNLP | November 2020
Meta-Learning For Few-Shot NMT Adaptation
Amr Sharaf, Hany Hassan Awadalla, Hal Daumé III
April 2020
Publications by Research Area
-
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
February 2023
Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production
Hany Hassan Awadalla, YOUNG JIN KIM
December 2022
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Rui Liu, Young Jin Kim, Alexandre Muzio, Hany Hassan Awadalla
ICML 2022 | July 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi, Shufang Xie, Tao Qin, Hany Hassan Awadalla
NAACL 2022 (The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies) | July 2022
Taming Sparsely Activated Transformer with Stochastic Experts
Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
ICLR 2022 | April 2022
Improving Multilingual Translation by Representation and Gradient Regularization
Yilin Yang, Akiko Eriguchi, Alexandre Muzio, Prasad Tadepalli, Stefan Lee, Hany Hassan Awadalla
EMNLP 2021 (The 2021 Conference on Empirical Methods in Natural Language Processing) | November 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task.
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
WMT | November 2021
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio, Felipe Cruz Salinas, Liyang Lu, Amr Hendy, Samyam Rajbhandari, Yuxiong He, Hany Hassan Awadalla
September 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
arXiv: Computation and Language | June 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
June 2021
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders.
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei
December 2020
Multi-task Learning for Multilingual Neural Machine Translation
Yiren Wang, Hany Hassan Awadalla
EMNLP | December 2020
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding
Young Jin Kim, Hany Hassan Awadalla
SustaiNLP – EMNLP | November 2020
Meta-Learning For Few-Shot NMT Adaptation
Amr Sharaf, Hany Hassan Awadalla, Hal Daumé III
April 2020
-
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
February 2023
Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production
Hany Hassan Awadalla, YOUNG JIN KIM
December 2022
Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPU
Hossam Amer, Young Jin Kim, Mohamed Afify, Hitokazu Matsushita, Hany Hassan Awadalla
AMTA | August 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
Muhammad ElNokrashy, Amr Hendy, Mohamed Maher, Mohamed Afify, Hany Hassan Awadalla
AMTA | August 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi, Shufang Xie, Tao Qin, Hany Hassan Awadalla
NAACL 2022 (The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies) | July 2022
Taming Sparsely Activated Transformer with Stochastic Experts
Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
ICLR 2022 | April 2022
Improving Multilingual Translation by Representation and Gradient Regularization
Yilin Yang, Akiko Eriguchi, Alexandre Muzio, Prasad Tadepalli, Stefan Lee, Hany Hassan Awadalla
EMNLP 2021 (The 2021 Conference on Empirical Methods in Natural Language Processing) | November 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task.
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
WMT | November 2021
Discovering Representation Sprachbund For Multilingual Pre-Training
Yimin Fan, Yaobo Liang, Alexandre Muzio, Hany Hassan Awadalla, Houqiang Li, Ming Zhou, Nan Duan
EMNLP 2021 | October 2021
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio, Felipe Cruz Salinas, Liyang Lu, Amr Hendy, Samyam Rajbhandari, Yuxiong He, Hany Hassan Awadalla
September 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
arXiv: Computation and Language | June 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
June 2021
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders.
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei
December 2020
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding
Young Jin Kim, Hany Hassan Awadalla
SustaiNLP – EMNLP | November 2020
Meta-Learning For Few-Shot NMT Adaptation
Amr Sharaf, Hany Hassan Awadalla, Hal Daumé III
April 2020
Publications by Type
-
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
February 2023
Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production
Hany Hassan Awadalla, YOUNG JIN KIM
December 2022
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
arXiv: Computation and Language | June 2021
-
Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPU
Hossam Amer, Young Jin Kim, Mohamed Afify, Hitokazu Matsushita, Hany Hassan Awadalla
AMTA | August 2022
Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
Muhammad ElNokrashy, Amr Hendy, Mohamed Maher, Mohamed Afify, Hany Hassan Awadalla
AMTA | August 2022
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Rui Liu, Young Jin Kim, Alexandre Muzio, Hany Hassan Awadalla
ICML 2022 | July 2022
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Akiko Eriguchi, Shufang Xie, Tao Qin, Hany Hassan Awadalla
NAACL 2022 (The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies) | July 2022
Taming Sparsely Activated Transformer with Stochastic Experts
Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan Awadalla, Ruofei Zhang, Tuo Zhao, Jianfeng Gao
ICLR 2022 | April 2022
Improving Multilingual Translation by Representation and Gradient Regularization
Yilin Yang, Akiko Eriguchi, Alexandre Muzio, Prasad Tadepalli, Stefan Lee, Hany Hassan Awadalla
EMNLP 2021 (The 2021 Conference on Empirical Methods in Natural Language Processing) | November 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task.
Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
WMT | November 2021
Discovering Representation Sprachbund For Multilingual Pre-Training
Yimin Fan, Yaobo Liang, Alexandre Muzio, Hany Hassan Awadalla, Houqiang Li, Ming Zhou, Nan Duan
EMNLP 2021 | October 2021
Multi-task Learning for Multilingual Neural Machine Translation
Yiren Wang, Hany Hassan Awadalla
EMNLP | December 2020
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding
Young Jin Kim, Hany Hassan Awadalla
SustaiNLP – EMNLP | November 2020
-
Scalable and Efficient MoE Training for Multitask Multilingual Models
Young Jin Kim, Ammar Ahmad Awan, Alexandre Muzio, Felipe Cruz Salinas, Liyang Lu, Amr Hendy, Samyam Rajbhandari, Yuxiong He, Hany Hassan Awadalla
September 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei
June 2021
-
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders.
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei
December 2020
Meta-Learning For Few-Shot NMT Adaptation
Amr Sharaf, Hany Hassan Awadalla, Hal Daumé III
April 2020