Springtime on the sun-drenched Amalfi Coast. A perfect little café perched high above the sea, the scent of jasmine and lemon blossoms wafting past. You open the menu, hungry for lunch. Oh, wait—you don’t know any Italian. Now what?
Not to worry: Just whip out your Windows Phone (opens in new tab), hover the camera over the menu, and the Bing Translator app (opens in new tab) will tell you what your choices are. Better yet, you can speak into the phone’s mic, and it will tell the waiter, “I’ll have the grilled squid”—in Italian. Don’t have a connection to the internet? No problem—by downloading the supported language packs before your trip, you can travel with confidence, even when an internet connection is not available.
It sounds improbably easy, and it is. The app for Windows Phone, a free download that works both online and offline, is just one example of how Microsoft’s decades of research into machine learning and translation are bearing fruit.
GigaPath: Whole-Slide Foundation Model for Digital Pathology
Digital pathology helps decode tumor microenvironments for precision immunotherapy. In joint work with Providence and UW, we’re sharing Prov-GigaPath, the first whole-slide pathology foundation model, for advancing clinical research.
As the world celebrates International Translation Day (opens in new tab) on Sept. 30, attention increasingly is turning to the ways that translation is occurring, on the web and on computers and mobile devices.
At Microsoft, the technology behind the company’s translation tools and capabilities from Office (opens in new tab) to Lync (opens in new tab) originate with the Machine Translation (opens in new tab) team—which started as an incubation within the Natural Language Processing (opens in new tab) group and which includes scientists, lexicographers, linguists, software engineers, and machine-learning experts.
For more than 20 years, Microsoft Research has been investing heavily in investigating how computers can learn from the massive amounts of data being generated by our online interactions, devices, services, social networks, and sensors—and how that data can be used to produce real-time language translations.
Vikram Dendi
“People speak different languages,” says Vikram Dendi (opens in new tab), who heads product strategy for the Machine Translation team, “and as much as the Internet has started connecting everybody, there is still this challenge of people feeling more comfortable with one language over another and many people who know just one language.”
True to his background as an entrepreneur with product and marketing experience, Dendi has helped turn a small research group into a vital incubator of translation tools and services that are integrated in a wide range of Microsoft products and used by an extensive number of partners, developers, and customers.
Unlike most leaders at Microsoft Research, Dendi is not a scientist or engineer.
“My role has been somewhat unorthodox from day one at Microsoft Research,” he says. “I’ve generally seen it as being the bridge between the excellent research work that we do and the kind of impact that technology could have on the world.”
Dendi has infused his group with a startup mentality, aiming for fast-paced innovation and an eye on “packaging up technology to get it to the right people and trying to be innovative at every layer”—what he calls “giving the technology its due.”
A Web-Based Translation Engine
At a time when Microsoft was just getting started in online services, the Machine Translation group made a prescient decision to make Microsoft’s machine-translation technology a cloud-based web service.
“There’s a big reason why we did that,” Dendi says. “Five or six years ago, when you thought about translation, you thought about going to BabelFish or Bing Translator and typing something in. In some cases, you didn’t know how to type it in because it was in a different language or a different script. We wanted to transform translation from being a destination to being an option seamlessly integrated into whatever natural action you were taking.”
Dendi’s ultimate goal is to have translation become “something that is just taken for granted, a technology that is seamlessly part of the fabric of the computing infrastructure that you use.” He thinks of its evolution like that of GPS.
“When GPS devices were introduced and people started carrying them around, it made some interesting changes to our lives,” he notes. “Today, we’re much more confident about going to places we’ve never been, because we know that there is a way for us to find our destination. In fact, we might even take the availability of such technology for granted. Similarly, I think translation will be a tool that will encourage you to venture forth with greater confidence and eventually become an integral and expected part of your communication experience. Like GPS, you will only notice it once it goes missing.”
Basing the Microsoft translation system in the cloud not only promotes that ubiquity but also has enabled Dendi’s team to innovate quickly. For example, the researchers realized that the translation system was having difficulty with some of the language used on Facebook, so they built and implemented a Social Text Corrector—a machine-learning-based correction technology to improve social-media translations.
The cloud model also enables the researchers to make faster use of the massive amounts of data generated by Bing (opens in new tab), which the researchers use to train the translation service.
Dendi’s team works in partnership with product groups at Microsoft—including Bing, Windows Phone, Windows 8 (opens in new tab), Internet Explorer (opens in new tab), Office, Yammer (opens in new tab), Lync, Visual Studio (opens in new tab), and SharePoint (opens in new tab). The resulting translation features include the various Bing Translator experiences, which support dozens of languages (including, for you Star Trek fans, Klingon); the Translation Accelerator in Internet Explorer, which lets you instantly translate text on any web page without leaving the page; and the new Translator App for Office, launching later today, which lets you translate directly within Office documents.
More than 80,000 developers are using the Microsoft translation engine, and they are taking advantage of built-in customization features to adapt the translation system to their content. More than 40 languages are supported, and the list is growing continually.
The implications of the technology are huge, with the potential to unlock new possibilities not only in commerce, travel, and education, but also in areas such as health care and government.
What’s Next
Dendi’s group, along with other researchers from Microsoft, is pushing the boundaries not only on text translation, but also in translation of human speech. Recently, Microsoft Research unveiled a promising breakthrough in speech-translation technology (opens in new tab) that combines novel speech-recognition technologies and a technique called “deep neural networks,” which is patterned after human brain behavior. The result is a dramatic improvement in accuracy.
It could be a long time before technology can match what human translators can do, but Dendi is confident that the goal is attainable.
“We’re nowhere close to human-level translation,” Dendi says. “But our users and customers are starting to see some significant quality gains in domain-specific customized translations built using the Microsoft Translator Hub (opens in new tab). We’ll continue to make the investment to keep pushing the quality forward and continue to do the experiments and the deep research needed for the next breakthrough. While our focus will be on the usefulness of the translation, I expect there will be significant progress towards achieving broad domain human-level translation within the next decade.”
Meanwhile, he and his team are relishing the challenge of solving theoretical problems that have immediate application in commercial products.
“My first-ever full-time job was my own startup,” Dendi says. “If you have interacted with people who have been in the entrepreneurial circles, one thing that is fairly common is a burning desire to try to keep making rapid, continuous progress.”
On International Translation Day, that’s the sort of attitude that holds promise for even more translation-related advances in the years to come.
Artificial intelligence 
Human language technologies