[PYTHON] Introduction to Machine Translation Architecture by the University of Cambridge by Slack Translation App Kiara

Introduction

I would like to introduce a paper from the University of Cambridge where the evolution of machine translation in 2019 is very well organized.

We have developed our own translation Chatbot Plugin for Slack, Kiara. Lead engineer Harada is working hard as Japan's first Slack Developer Chapter Leader. https://kiara-app.com/ (Free trial version available) With a passion for Slack and the work style revolution, we will continue to excite the developer community.

Abstract

Neural Machine Translation: A Review

(Submitted on 4 Dec 2019) The field of machine translation (MT), the automatic translation of written text from one natural language into another, has experienced a major paradigm shift in recent years. Statistical MT, which mainly relies on various count-based models and which used to dominate MT research for decades, has largely been superseded by neural machine translation (NMT), which tackles translation with a single neural network. In this work we will trace back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family. We will conclude with a survey of recent trends in the field.

The field of machine translation (MT), the automatic translation of sentences from one natural language to another, has undergone a major paradigm shift in recent years. Statistical MT, which relied primarily on various count-based models and dominated MT research for decades, has been largely replaced by Neural Machine Translation (NMT), which works on translation with a single neural network. This task traces the origins of the latest NMT architectures to word and sentence embeddings and previous examples of encoder / decoder network families. Finally, we will investigate recent trends in this area.

https://arxiv.org/abs/1912.02047

Conclusion

Neural machine translation (NMT) has become the de facto standard for large-scale machine translation in a very short period of time. This article traced back the origin of NMT to word and sentence embeddings and neural language models. We reviewed the most commonly used building blocks of NMT architectures – recurrence, convolution, and attention – and discussed popular concrete architectures such as RNNsearch, GNMT, ConvS2S, and the Transformer. We discussed the advantages and disadvantages of several important design choices that have to be made to design a good NMT system with respect to decoding, training, and segmentation. We then explored advanced topics in NMT research such as explainability and data sparsity.

Main Topic

NMT=Neural Machine Translation

Word Embeddings

Phrase Embeddings

Sentence Embeddings

Encoder-Decoder Networks

Attentional Encoder-Decoder Networks

Recurrent NMT

Convolutional NMT

Self attention based NMT

Search problem in NMT

Greedy and beam search

Decoding direction

Generating diverse translation

Simultaneous translation

Open vocabulary NMT

NMT model errors

Reinforcement learning

Adversarial training

Explainable NMT

Multilingual NMT

Screen Shot 2020-01-12 at 21.11.29.png Screen Shot 2020-01-12 at 21.14.47.png Screen Shot 2020-01-12 at 21.15.47.png Screen Shot 2020-01-12 at 21.17.22.png

Recommended Posts

Introduction to Machine Translation Architecture by the University of Cambridge by Slack Translation App Kiara
[Translation] scikit-learn 0.18 Tutorial Introduction of machine learning by scikit-learn
A quick introduction to the neural machine translation library
Introduction to Statistics The University of Tokyo Press Chapter 2 Exercises
The format of the message obtained by Slack API is subtly difficult to use
From the introduction of pyethapp to the execution of contract
Introduction to machine learning ~ Let's show the table of K-nearest neighbor method ~ (+ error handling)
Add a function to tell the weather of today to slack bot (made by python)
[Introduction to Python] Basic usage of the library matplotlib
I tried to verify the yin and yang classification of Hololive members by machine learning
Introduction to machine learning
Try to evaluate the performance of machine learning / regression model
Predict the presence or absence of infidelity by machine learning
Try to evaluate the performance of machine learning / classification model
How to increase the number of machine learning dataset images
Debug by attaching to the Python process of the SSH destination
[Machine learning] I tried to summarize the theory of Adaboost