Facebook Transitioning to Neural Machine Translation

| by Alex Giamas Follow 9 Followers on Aug 17, 2017. Estimated reading time: 1 minute |

Facebook recently announced the global rollout of NMT (Neural Machine Translation). Switching from phrase based translation models to NMT has been rolled out for more than 2,000 translation directions and 4.5 billion translations per day. According to Facebook this provides an 11% increase in BLEU score, a well known scoring mechanism for translation scoring.

The engineers behind this blog post Juan Miguel Pino, Alexander Sidorov and Necip Fazil Ayan achieved a 2.5X boost in performance using Caffe2, the Machine Learning framework Facebook open sourced earlier this year.

The key to achieving this level of translation quality lies in using a recurrent neural network known as sequence-to-sequence LSTM (long short-term memory) with attention. A system like this takes into account the context of the source sentence and everything preceding it to generate more accurate translations. This allows for long-distance reordering in translation, solving a problem found amongst others between some pairs of translation like English-Turkish and English-Greek.

The attention mechanism of LSTM also helps when handling with unknown words in translations by soft aligning translations into a bilingual lexicon built from the training data. Vocabulary reduction is used to to reduce computation time significantly without notably affecting translation quality.

The FB Learner Flow framework was used to fine-tune parameters for each of the 2,000 translation pairs quickly and accurately. This alone contributed to a 3.7% increase in BLEU score for English-Spanish translation pairs.

As part of this project, Facebook contributed and open sourced Recurrent Neural Networks into Caffe2. Facebook’s Artificial Intelligence Research (FAIR) team has published its research on using convolutional neural networks (CNNs) for machine translation as part of this project as well.

Facebook’s announcement comes a few months after Google announced NMT for Google Translate back in 2016 and less than a month after Google announced its Multimodel Neural Network breakthrough.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread


Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you