Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu Wei
MSR-TR-2024-42 | October 2024
Published by Microsoft
Yi Zhu
SENIOR RESEARCH SDE
Publications
Publications by Year
-
nnScaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training
Zhiqi Lin, Youshan Miao, Quanlu Zhang, Fan Yang, Yi Zhu, Cheng Li, Saeed Maleki, Xu Cao, Ning Shang, Yilei Yang, Weijiang Xu, Mao Yang, Lintao Zhang, Lidong Zhou
OSDI | July 2024
You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei
NeurIPS 2024 | May 2024
-
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023
Publications by Research Area
-
Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu Wei
MSR-TR-2024-42 | October 2024
Published by Microsoft
nnScaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training
Zhiqi Lin, Youshan Miao, Quanlu Zhang, Fan Yang, Yi Zhu, Cheng Li, Saeed Maleki, Xu Cao, Ning Shang, Yilei Yang, Weijiang Xu, Mao Yang, Lintao Zhang, Lidong Zhou
OSDI | July 2024
You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei
NeurIPS 2024 | May 2024
-
nnScaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training
Zhiqi Lin, Youshan Miao, Quanlu Zhang, Fan Yang, Yi Zhu, Cheng Li, Saeed Maleki, Xu Cao, Ning Shang, Yilei Yang, Weijiang Xu, Mao Yang, Lintao Zhang, Lidong Zhou
OSDI | July 2024
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023
Publications by Type
-
Differential Transformer
Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu Wei
MSR-TR-2024-42 | October 2024
Published by Microsoft
-
nnScaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training
Zhiqi Lin, Youshan Miao, Quanlu Zhang, Fan Yang, Yi Zhu, Cheng Li, Saeed Maleki, Xu Cao, Ning Shang, Yilei Yang, Weijiang Xu, Mao Yang, Lintao Zhang, Lidong Zhou
OSDI | July 2024
You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei
NeurIPS 2024 | May 2024
Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
Sixth Conference on Machine Learning and Systems (MLSys'23) | June 2023