AI at Scale
Models, infrastructure and hardware for next-generation AI applications
新闻与深度文章
新闻报道 | New York Times
You Can’t Spell Creative Without A.I.
Advances in software applications that process human language lie at the heart of the debate over whether computer technologies will enhance or even substitute for human creativity.
Rangan Majumder is the Partner Group Program Manager of Microsoft’s Search and AI, and he has a simple goal: to make the world smarter and more productive. But nobody said simple was easy, so he and his team are working…
| Corby Rosset
This figure was adapted from a similar image published in DistilBERT. Turing Natural Language Generation (T-NLG) is a 17 billion parameter language model by Microsoft that outperforms the state of the art on many downstream NLP tasks. We present a…
ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters
| DeepSpeed Team, Rangan Majumder, 和 Junhua Wang
The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly…
新闻报道 | Fortune
A.I. and tackling the risk of “digital redlining”
Last week, a Dutch court ordered the government in the Netherlands to stop using a machine-learning algorithm for detecting welfare fraud, citing human rights violations. The system, called System Risk Indicator (SyRI) in English, was being used by four Dutch…
Transformer-based language generation models have enabled better conversational applications. Though they still have their shortcomings, which were recently exposed by a team at MIT, researchers continue improving them to build better, larger, and more robust models.
新闻报道 | WinBuzzer
Microsoft’s New Turing NLG is the Largest Transformer Language Model
Microsoft has developed a Transformer-based language generation model that it describes as the largest ever made. This week, Microsoft AI & Research announced Turing NLG, which is twice the size of its nearest competitor.
Microsoft has released a new open-source library called DeepSpeed, which, when combined with its ‘ZeRO’ module can train 100 billion parameter models without using the resources traditionally associated with that.
Microsoft has revealed its largest deep learning language model, the Turing Natural Language Generation (T-NLG), which is claimed to have a record-breaking 17 billion parameters. The T-NLG, according to Microsoft, outperforms the largest deep learning models to date: the University of Washington’s Grover-Mega and Nvidia’s MegatronLM, which…