Skip to main content
Translator

Microsoft Translator Blog

Microsoft Custom Translator pushes the translation quality bar closer to human parity

Posted on by Microsoft Translator

The Custom Translator journey to be on the leading edge of machine translation technology continues.

In early August 2020, we started our Custom Translator upgrade from Long Short-Term Memory (aka LSTM) based neural machine translation architecture (or V1) to our Microsoft Translator’s state-of-the-art Transformer based architecture (or V2). V2 is the same translation architecture which powers the standard uncustomized Translator API, as well as translation in Microsoft Office 365, Speech, Bing.com/translator, Edge, and more.

The August release enabled customers to use the dictionary (phrase or sentence) document type to build custom models on top of the V2 platform for a quick translation quality improvement over the V1 platform.

Today, Custom Translator completed the full V2 platform upgrade to deliver an even bigger translation quality gain than before. Customers can now build custom models with all document types (Training, Testing, Tuning, Phrase Dictionary and Sentence Dictionary) using full text documents, like Office documents, PDFs, HTML and plain text.

With this release, enterprises, small and medium sized businesses, app developers, and language service providers can build advanced custom neural translation systems that respect their defined business terminology and seamlessly integrate those systems into existing or new applications, workflows, and websites to attract customers and grow the business.

We put every new baseline language model through a rigorous human evaluation process to ensure the translation quality continues to meet high standards on generic input across all domains. However, with custom trained specialized translation systems, customers can achieve much higher adherence to the domain-specific terminology and style by training a custom translation system on previously translated, in-domain documents. These previously translated documents allow Custom Translator to learn the preferred translations in context, so Translator can apply these terms and phrases when the context calls for it, and produce a fluent translation in the target language, respecting the context-dependent grammar of the target language.

Benefits of the upgrade

We use BLEU score (a standard way in the research community) to measure the translation quality of a newly trained baseline model. A one or two BLEU point gain is a worthy achievement. The Custom Translator V2 platform upgrade will deliver significant improvements when compared to the previous V1 platform. The bar chart below depicts the translation quality BLEU score improvement for some domains and the impact of training dataset size.

Sample domains translation quality BLEU score when using standard Translator, Custom Translator V1, and Custom Translator V2.
Training dataset size in thousands (‘auto-28k’ means 28,000 parallel sentences for the automotive domain)

It is important to note that actual quality improvement is dependent upon customer data quality, training dataset size, and domain coverage.

“We’re hoping that translation through a neural network will not only boost quality and speed, but also offer advances in the evaluation of big data,” said Nikolas Meyer-Aun, Head of Quality and Supplier Management for Languages at Volkswagen AG

Custom Translator will offer FREE upgrade to V2. You can retrain one model per project in a workspace to the V2 platform at no charge. When you view a project, you should see a message if you still have free upgrade credit for that project. The offer starts today and ends on January 31, 2021. After January 31, 2021, normal training charges apply for each retraining.

You can learn more about the upgrade to version 2 in the FAQ below.

FAQ

  1. What are you releasing in V2?
    We are releasing a Custom Translator platform upgrade (V2) to deliver significant translation quality improvements using Microsoft’s state-of-the-art neural machine translation architecture. The user experience remains the same as in V1. There is nothing new to learn, just better translations. For a quick refresher, please watch this Quick-Start video on YouTube.
  2. What is the benefit of upgrading a currently deployed model to the V2 platform?
    Significantly improved translation quality; the Custom Translator V2 platform upgrade will deliver significant improvement over standard Translator and the previous V1 platform. The bar chart above shows the translation quality BLEU scores for some common domains and the impact of training dataset size.
  3. How do I use my FREE upgrade to V2 credit?
    When you view a project details, a free credit message becomes visible (see images below) and will continue to be visible until you either successfully retrain a model or the offer ends on January 31, 2021.

  4. How do I upgrade my deployed (or undeployed) custom models to the V2 platform?
    If you have a deployed or undeployed dictionary-only model that was retrained after August 3, 2020, it has already been upgraded to the V2 platform. Otherwise, you should launch a new model training. Once the new model is successfully trained, the “Swap” button will be visible. Click “Swap” to deploy the new model. Note: “Swap” button enables no-downtime deployment. That is, all translation requests will continue to be served by the previous model until the new model is deployed and functional, then the new model will serve new translation requests and the previous model will be undeployed.
  5. Can I continue to re-deploy V1 custom models after the V2 platform upgrade?
    Yes. We understand there may be reasons customers would want to continue using models trained on the V1 platform. V1 models can run on the V2 platform. In the future, V1 models will not be re-deployable. Note: Once you retrain a V1 model, the new model will be on the V2 platform. The “Model” tab will show all new (V2) and previous (V1) models.
  6. I’m new to customizing translations, how do I get started using Custom Translator?
    To build a custom model, watch the Quick-Start video above and refer to the documentation on docs.microsoft.com: https://docs.microsoft.com/azure/cognitive-services/translator/custom-translator/overview

What you can do with Microsoft Custom Translator

Build custom models with your domain specific terminology and translate real-time using the Microsoft Translator API.

Use Microsoft Custom Translator with your translation solutions to help globalize your business and improve customer interactions.

For more information on Microsoft Translator solutions please visit: https://www.microsoft.com/translator/business.