Unsupervised Machine Translation Using Monolingual Corpora Only: Difference between revisions

From statwiki
Jump to navigation Jump to search
Line 12: Line 12:
= Methodology =
= Methodology =


The objective function that proposed by the paper is a combination of 3 component objective functions:
# Reconstruction loss of the denoising auto-encoder
# Cross domain loss of the auto-encoder
# Adversarial cross entropy loss of the discriminator


==Denoising Auto-Encoding==
 
===Reconstruction Loss===
 
\begin{align}
\mathcal{L}_{auto}(\theta_{enc}, \theta_{dec}, \mathcal{Z}, \ell) = E_{x\sim D_\ell, \hat{x}\sim d(e(C(x),\ell),\ell)}[\Delta(\hat{x},x)]
\end{align}
 
===Cross Domain Training===
 
 
===Adversarial Training===


= Critique =
= Critique =

Revision as of 17:10, 19 November 2018

Introduction

The paper presents an unsupervised method to machine translation using only monoligual corpora without any alignment between sentences or documents. Monoligual corpora are text corpora that is made up of one language only. This contrasts with the usual translation approach that uses parallel corpora, where two corpora are the direct translation of each other and the translations are aligned by words or sentences.

The general approach of the methodology is to first use a unsupervised word-by-word translation model proposed by [Conneau, 2017], then iteratively improve on the translation by utilizing 2 architectures:

  1. A denoising auto-encoder to reconstruct noisy versions of sentences for both source and target languages.
  2. A discriminator to align the distributions of the source and target languages in a latent space.

Background

Methodology

The objective function that proposed by the paper is a combination of 3 component objective functions:

  1. Reconstruction loss of the denoising auto-encoder
  2. Cross domain loss of the auto-encoder
  3. Adversarial cross entropy loss of the discriminator


Reconstruction Loss

\begin{align} \mathcal{L}_{auto}(\theta_{enc}, \theta_{dec}, \mathcal{Z}, \ell) = E_{x\sim D_\ell, \hat{x}\sim d(e(C(x),\ell),\ell)}[\Delta(\hat{x},x)] \end{align}

Cross Domain Training

Adversarial Training

Critique


Other Sources

References

  1. [Lample, 2018] Lample, G., Conneau, A., Ranzato, M., Denoyer, L., "Unsupervised Machine Translation Using Monolingual Corpora Only". arXiv:1711.00043
  1. [Conneau, 2017] Conneau, A., Lample, G., Ranzato, M., Denoyer, L., Jégou, H., "Word Translation without Parallel Data". arXiv:1710.04087