Hence, we make sure the encoder encodes sentences from source and target languages to the same latent space. This ensures the decoder can translate regardless of the input source language. We perform this enforcement by adversarial training following (Lample et al., 2018a) where we constrain the encoder to map the two languages to the same feature space. We do this by training a discriminator to classify encodings of source and target sentences. We also make sure the same latent space is used for both language modelling and translation so that the language model can be transferred nicely to the translation task.Īt each training step, we perform the following: The encoder is trained to fool the discriminator such that latent representations of both source or target are indistinguishable.
