View Project

Title:

A Teacher-Student Framework for Zero-Resource Neural Machine Translation

Abstract:

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentence shave close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model (“student”) without parallel corpora available, guided by an existing pivot-to-target NMT model (“teacher”) on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.

Comments:

Actions

Suggested Topics

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Highlights of EMNLP 2017: Exciting Datasets, Return of the Clusters, and More! Sebastian Ruder 641 resource 180.88
2 NLP’s generalization problem, and how researchers are tackling it Ana Marasovic 711 resource 166.04
3 Codra: A Novel Discriminative Framework for Rhetorical Analysis Shafiq Joty, Giuseppe Carenini, Raymond T. Ng 999 paper 164.62
4 Introduction to Neural Machine Translation with GPUs (part 3) Kyunghyun Cho 753 tutorial 164.23
5 Similarity-driven Semantic Role Induction via Graph Partitioning Joel Lang, Mirella Lapata 999 paper 162.32
6 ACL 2017 Report Yuta Kikuchi, Sosuke Kobayashi 711 resource 157.86
7 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek K 713 resource 157.20
8 Neural Machine Translation (seq2seq) Tutorial Thang Luong, Eugene Brevdo, Rui Zhao 753 tutorial 150.53
9 NLP's ImageNet moment has arrived Sebastian Ruder 862 resource 150.08
10 ICML+ACL’18: Structure Back in Play, Translation Wants More Context Andre Martins 956 resource 149.45
11 Deep Learning for NLP, advancements and trends in 2017 Javier 711 resource 148.85
12 Tips on Building Neural Machine Translation Systems Graham Neubig 753 tutorial 146.74
13 Discriminative Syntax-based Word Ordering for Text Generation Yue Zhang, Stephen Clark 999 paper 145.94
14 A survey of transfer learning Karl Weiss, Taghi M. Khoshgoftaar and DingDing Wang 978 resource 139.48
15 What I learned from Deep Learning Summer School 2016 Hamid Palangi 107 tutorial 139.27
16 A survey of cross-lingual embedding models Sebastian Ruder 721 tutorial 138.65
17 DEEP LEARNING FOR CHATBOTS, PART 1 - INTRODUCTION Denny Britz 445 tutorial 138.58
18 Recurrent Neural Networks Tutorial, Part 2 - Implementing a RNN with Python, Numpy, and Theano Denny Britz 741 tutorial 137.92
19 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 978 tutorial 137.91
20 Train Neural Machine Translation Models with Sockeye Felix Hieber, Tobias Domhan 753 tutorial 137.23
21 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 421 survey 135.67
22 A miscellany of fun deep learning papers Adrian Colyer 711 resource 135.23
23 K-Means & Other Clustering Algorithms: A Quick Intro with Python Nikos Koufos 571 tutorial 134.97
24 Introduction to Neural Machine Translation with GPUs (Part 2) Kyunghyun Cho 753 tutorial 134.28
25 DEEP LEARNING FOR CHATBOTS, PART 2 - IMPLEMENTING A RETRIEVAL-BASED MODEL IN TENSORFLOW Denny Britz 445 tutorial 133.91
26 Introduction to Neural Machine Translation with GPUs (part 1) Kyunghyun Cho 753 tutorial 133.21
27 Unsupervised machine translation: A novel approach to provide fast, accurate translations for more languages MarcAurelio Ranzato, Guillaume Lample, Myle Ott 956 resource 132.26
28 Recurrent Neural Networks Tutorial, Part 1 - Introduction to RNNs Denny Britz 741 tutorial 131.28
29 Negated bio-events: analysis and identification Raheel Nawaz, Paul Thompson, Sophia Ananiadou 999 paper 130.60
30 How do we capture structure in relational data? Matthew Das Sarma 711 resource 130.57
31 The history and meaning of the journal impact factor Eugene Garfield 999 paper 130.49
32 Deep Learning in NLP Vered Shwartz 711 resource 130.06
33 Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience Ismael Rafols, Martin Meyer 999 paper 130.04
34 The h’-Index, Effectively Improving the h-Index Based on the Citation Distribution Chun-Ting Zhang 999 paper 129.34
35 Recurrent Neural Networks Tutorial, Part 3- Backpropagation Through Time and Vanishing Gradients Denny Britz 741 tutorial 126.51
36 Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano Denny Britz 742 tutorial 126.28
37 A history of machine translation from the Cold War to deep learning Ilya Pestov 753 resource 125.63
38 PyTorch-GAN Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, Eli Shechtman 731 library 125.47
39 The data that transformed AI research—and possibly the world Dave Gershgorn 107 resource 124.60
40 Gimli: open source and high-performance biomedical name recognition David Campos, Sergio Matos, Jose Oliveira 999 paper 124.29
41 An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship J. E. Hirsch 999 paper 123.59
42 Attention and Memory in Deep Learning and NLP Denny Britz 745 tutorial 122.64
43 On word embeddings - Part 2: Approximating the Softmax Sebastian Ruder 721 tutorial 122.54
44 Summaries and notes on Deep Learning research papers Denny Britz 713 resource 122.29
45 Word embeddings in 2017: Trends and future directions Sebastian Ruder 721 resource 122.08
46 The 7 NLP Techniques That Will Change How You Communicate in the Future (Part I) James Le 112 resource 121.85
47 Four deep learning trends from ACL 2017: Part 1 Abigail See 713 resource 121.00
48 Do Altmetrics Work? Twitter and Ten Other Social Web Services Mike Thelwall, Stefanie Haustein, Vincent Larivière, Cassidy R. Sugimoto 999 paper 120.83
49 Simple, Strong Deep-Learning Baselines for NLP in several frameworks Dan Pressel 713 library 120.68
50 Rohan & Lenny #3: Recurrent Neural Networks & LSTMs Rohan Kapur 741 tutorial 120.66
51 The Annotated Transformer Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, Illia Polosukhin 745 library 120.32
52 Analyzing the Meaning of Sentences Steven Bird, Ewan Klein, Edward Loper 721 course 119.89
53 Deep Learning for NLP Best Practices Sebastian Ruder 713 tutorial 119.19
54 Machine Learning for Humans Vishal Maini, Samer Sabri 134 tutorial 119.11
55 Deep Learning 2: Part 2 Lesson 11 Hiromi Suenaga 711 resource 118.60
56 Multi-Task Learning Objectives for Natural Language Processing Author Unknown 133 resource 118.54
57 An Overview of Multi-Task Learning in Deep Neural Networks Sebastian Ruder 829 tutorial 117.98
58 Tombones Computer Vision Blog Tomasz Malisiewicz 958 resource 117.83
59 A Complete Tutorial to Learn Data Science with Python from Scratch Kunal Jain 131 tutorial 117.57
60 SentencePiece Taku Kudo 432 library 116.48
61 An overview of gradient descent optimization algorithms Sebastian Ruder 187 tutorial 116.17
62 An overview of gradient descent optimization algorithms Sebastian Ruder 187 resource 115.65
63 LDA2vec: Word Embeddings in Topic Models Lars Hulstaert 721 resource 115.28
64 ATTENTION AND MEMORY IN DEEP LEARNING AND NLP Denny Britz 745 tutorial 115.08
65 Semi-Supervised Learning for Neural Machine Translation Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu 999 paper 114.82
66 Requests for Research Sebastian Ruder 921 resource 114.70
67 A Beginner’s Guide to Deep Reinforcement Learning Adam Gibson, Chris Nicholson, Josh Patterson 857 library 114.64
68 An Overview of Proxy-label Approaches for Semi-supervised Learning Sebastian Ruder 581 resource 113.30
69 Learning when to skim and when to read Alexander Rosenberg Johansen, Bryan McCann, James Bradbury, Richard Socher 713 tutorial 113.24
70 Neural Networks Tutorial – A Pathway to Deep Learning Andy Thomas 711 tutorial 113.14
71 Rohan #2: Artificial intelligence, ?Progress/?Time Rohan Kapur 811 tutorial 112.77
72 Minibatch Metropolis-Hastings Daniel Seita 107 tutorial 112.71
73 Peeking into the neural network architecture used for Google's Neural Machine Translation Stephen Merity 753 tutorial 111.46
74 Text Segmentation based on Semantic Word Embeddings Alexander Alemi, Paul Ginsparg 721 library 111.32
75 Gensim integration with scikit-learn and Keras Chinmaya Pancholi 713 library 111.16
76 Sequence-to-Sequence Learning with Attentional Neural Networks Guillaume Klein 753 library 110.89
77 New Theory Cracks Open the Black Box of Deep Learning Natalie Wolchover 811 resource 110.88
78 Simple Beginner’s guide to Reinforcement Learning & its implementation Faizan Shaikh 713 tutorial 110.57
79 What is machine learning? Everything you need to know Nick Heath 711 resource 110.16
80 Recursive Neural Networks with PyTorch James Bradbury 743 tutorial 109.46
81 Comprehensive Guide on t-SNE algorithm with implementation in R & Python SAURABH.JAJU2 341 tutorial 108.83
82 The Future (and Present) of Artificial Intelligence AMA Various Authors 811 resource 108.23
83 nmtpytorch lium-lst 731 library 107.93
84 Zoph_RNN: A C++/CUDA toolkit for training sequence and sequence-to-sequence models across multiple GPUs Xing Shi 741 library 107.77
85 Neural text generation: How to generate text using conditional language models Neil Yager 43 resource 107.42
86 Deep Learning for Computer Vision - Introduction to Convolution Neural Networks Aarshay Jain 744 tutorial 106.97
87 Deep Text Corrector Alex Paino 960 library 106.83
88 NEMATUS: Attention-based encoder-decoder model for neural machine translation Rico Sennrich 753 library 106.77
89 Transfer Learning for Low-Resource Neural Machine Translation Barret Zoph, Deniz Yuret, Jonathan May, Kevin Knight 999 paper 106.64
90 Shared Task: Machine Translation of News Author Unknown 131 resource 106.54
91 Written Memories: Understanding, Deriving and Extending the LSTM R2RT 742 resource 105.51
92 A Neural Network for Machine Translation, at Production Scale Quoc V. Le, Mike Schuster 753 resource 105.25
93 Normalizing Flows Tutorial, Part 2: Modern Normalizing Flows Eric Jang 811 resource 105.11