View Project

Title:

A Teacher-Student Framework for Zero-Resource Neural Machine Translation

Abstract:

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentence shave close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model (“student”) without parallel corpora available, guided by an existing pivot-to-target NMT model (“teacher”) on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.

Comments:

Actions

Suggested Topics

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Highlights of EMNLP 2017: Exciting Datasets, Return of the Clusters, and More! Sebastian Ruder 641 resource 181.91
2 Codra: A Novel Discriminative Framework for Rhetorical Analysis Shafiq Joty, Giuseppe Carenini, Raymond T. Ng 999 paper 165.62
3 Introduction to Neural Machine Translation with GPUs (part 3) Kyunghyun Cho 753 tutorial 165.41
4 Similarity-driven Semantic Role Induction via Graph Partitioning Joel Lang, Mirella Lapata 999 paper 163.33
5 ACL 2017 Report Yuta Kikuchi, Sosuke Kobayashi 711 resource 159.14
6 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek K 713 resource 158.32
7 Neural Machine Translation (seq2seq) Tutorial Thang Luong, Eugene Brevdo, Rui Zhao 753 tutorial 151.60
8 Deep Learning for NLP, advancements and trends in 2017 Javier 711 resource 149.97
9 Tips on Building Neural Machine Translation Systems Graham Neubig 753 tutorial 147.90
10 Discriminative Syntax-based Word Ordering for Text Generation Yue Zhang, Stephen Clark 999 paper 146.72
11 A survey of transfer learning Karl Weiss, Taghi M. Khoshgoftaar and DingDing Wang 978 resource 141.30
12 What I learned from Deep Learning Summer School 2016 Hamid Palangi 107 tutorial 140.26
13 A survey of cross-lingual embedding models Sebastian Ruder 721 tutorial 139.57
14 DEEP LEARNING FOR CHATBOTS, PART 1 - INTRODUCTION Denny Britz 445 tutorial 139.34
15 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 978 tutorial 139.02
16 Recurrent Neural Networks Tutorial, Part 2 - Implementing a RNN with Python, Numpy, and Theano Denny Britz 741 tutorial 138.74
17 Train Neural Machine Translation Models with Sockeye Felix Hieber, Tobias Domhan 753 tutorial 138.08
18 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 421 survey 136.58
19 A miscellany of fun deep learning papers Adrian Colyer 711 resource 136.24
20 K-Means & Other Clustering Algorithms: A Quick Intro with Python Nikos Koufos 571 tutorial 135.45
21 Introduction to Neural Machine Translation with GPUs (Part 2) Kyunghyun Cho 753 tutorial 135.22
22 DEEP LEARNING FOR CHATBOTS, PART 2 - IMPLEMENTING A RETRIEVAL-BASED MODEL IN TENSORFLOW Denny Britz 445 tutorial 134.66
23 Introduction to Neural Machine Translation with GPUs (part 1) Kyunghyun Cho 753 tutorial 134.14
24 Recurrent Neural Networks Tutorial, Part 1 - Introduction to RNNs Denny Britz 741 tutorial 132.02
25 Negated bio-events: analysis and identification Raheel Nawaz, Paul Thompson, Sophia Ananiadou 999 paper 131.55
26 The history and meaning of the journal impact factor Eugene Garfield 999 paper 131.01
27 Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience Ismael Rafols, Martin Meyer 999 paper 130.77
28 The h’-Index, Effectively Improving the h-Index Based on the Citation Distribution Chun-Ting Zhang 999 paper 130.05
29 Recurrent Neural Networks Tutorial, Part 3- Backpropagation Through Time and Vanishing Gradients Denny Britz 741 tutorial 127.22
30 Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano Denny Britz 742 tutorial 127.03
31 A history of machine translation from the Cold War to deep learning Ilya Pestov 753 resource 126.59
32 PyTorch-GAN Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, Eli Shechtman 731 library 126.41
33 The data that transformed AI research—and possibly the world Dave Gershgorn 107 resource 125.35
34 Gimli: open source and high-performance biomedical name recognition David Campos, Sergio Matos, Jose Oliveira 999 paper 125.23
35 An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship J. E. Hirsch 999 paper 124.23
36 Attention and Memory in Deep Learning and NLP Denny Britz 745 tutorial 123.61
37 On word embeddings - Part 2: Approximating the Softmax Sebastian Ruder 721 tutorial 123.39
38 Summaries and notes on Deep Learning research papers Denny Britz 713 resource 123.21
39 Word embeddings in 2017: Trends and future directions Sebastian Ruder 721 resource 122.98
40 The 7 NLP Techniques That Will Change How You Communicate in the Future (Part I) James Le 112 resource 122.86
41 Four deep learning trends from ACL 2017: Part 1 Abigail See 713 resource 122.01
42 Rohan & Lenny #3: Recurrent Neural Networks & LSTMs Rohan Kapur 741 tutorial 121.60
43 Do Altmetrics Work? Twitter and Ten Other Social Web Services Mike Thelwall, Stefanie Haustein, Vincent Larivière, Cassidy R. Sugimoto 999 paper 121.55
44 Simple, Strong Deep-Learning Baselines for NLP in several frameworks Dan Pressel 713 library 121.47
45 The Annotated Transformer Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, Illia Polosukhin 745 library 121.24
46 Analyzing the Meaning of Sentences Steven Bird, Ewan Klein, Edward Loper 721 course 120.81
47 Machine Learning for Humans Vishal Maini, Samer Sabri 134 tutorial 120.14
48 Deep Learning for NLP Best Practices Sebastian Ruder 713 tutorial 120.09
49 Multi-Task Learning Objectives for Natural Language Processing Author Unknown 133 resource 119.48
50 Deep Learning 2: Part 2 Lesson 11 Hiromi Suenaga 711 resource 119.47
51 A Complete Tutorial to Learn Data Science with Python from Scratch Kunal Jain 131 tutorial 119.32
52 An Overview of Multi-Task Learning in Deep Neural Networks Sebastian Ruder 829 tutorial 118.88
53 Tombones Computer Vision Blog Tomasz Malisiewicz 958 resource 118.79
54 SentencePiece Taku Kudo 432 library 117.20
55 An overview of gradient descent optimization algorithms Sebastian Ruder 187 tutorial 117.08
56 LDA2vec: Word Embeddings in Topic Models Lars Hulstaert 721 resource 117.00
57 An overview of gradient descent optimization algorithms Sebastian Ruder 187 resource 116.54
58 ATTENTION AND MEMORY IN DEEP LEARNING AND NLP Denny Britz 745 tutorial 115.96
59 Semi-Supervised Learning for Neural Machine Translation Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu 999 paper 115.78
60 Requests for Research Sebastian Ruder 921 resource 115.61
61 A Beginner’s Guide to Deep Reinforcement Learning Adam Gibson, Chris Nicholson, Josh Patterson 857 library 115.46
62 An Overview of Proxy-label Approaches for Semi-supervised Learning Sebastian Ruder 581 resource 114.21
63 Neural Networks Tutorial – A Pathway to Deep Learning Andy Thomas 711 tutorial 114.16
64 Learning when to skim and when to read Alexander Rosenberg Johansen, Bryan McCann, James Bradbury, Richard Socher 713 tutorial 113.99
65 Minibatch Metropolis-Hastings Daniel Seita 107 tutorial 113.65
66 Rohan #2: Artificial intelligence, ?Progress/?Time Rohan Kapur 811 tutorial 113.61
67 Peeking into the neural network architecture used for Google's Neural Machine Translation Stephen Merity 753 tutorial 112.38
68 Gensim integration with scikit-learn and Keras Chinmaya Pancholi 713 library 112.06
69 Text Segmentation based on Semantic Word Embeddings Alexander Alemi, Paul Ginsparg 721 library 112.05
70 Sequence-to-Sequence Learning with Attentional Neural Networks Guillaume Klein 753 library 111.66
71 Simple Beginner’s guide to Reinforcement Learning & its implementation Faizan Shaikh 713 tutorial 111.53
72 New Theory Cracks Open the Black Box of Deep Learning Natalie Wolchover 811 resource 111.49
73 What is machine learning? Everything you need to know Nick Heath 711 resource 111.08
74 Recursive Neural Networks with PyTorch James Bradbury 743 tutorial 110.25
75 Comprehensive Guide on t-SNE algorithm with implementation in R & Python SAURABH.JAJU2 341 tutorial 109.74
76 The Future (and Present) of Artificial Intelligence AMA Various Authors 811 resource 109.36
77 nmtpytorch lium-lst 731 library 108.69
78 Zoph_RNN: A C++/CUDA toolkit for training sequence and sequence-to-sequence models across multiple GPUs Xing Shi 741 library 108.60
79 Neural text generation: How to generate text using conditional language models Neil Yager 43 resource 108.32
80 Deep Learning for Computer Vision - Introduction to Convolution Neural Networks Aarshay Jain 744 tutorial 107.96
81 Deep Text Corrector Alex Paino 960 library 107.64
82 Transfer Learning for Low-Resource Neural Machine Translation Barret Zoph, Deniz Yuret, Jonathan May, Kevin Knight 999 paper 107.49
83 NEMATUS: Attention-based encoder-decoder model for neural machine translation Rico Sennrich 753 library 107.47
84 Shared Task: Machine Translation of News Author Unknown 131 resource 107.20
85 Written Memories: Understanding, Deriving and Extending the LSTM R2RT 742 resource 106.35
86 A Neural Network for Machine Translation, at Production Scale Quoc V. Le, Mike Schuster 753 resource 106.02
87 Deep Learning 2: Part 2 Lesson 13 Hiromi Suenaga 711 resource 104.91
88 Lexicalization and Generative Power in Ccg Marco Kuhlmann, Alexander Koller, Giorgio Satta 999 paper 104.08
89 Trends in Neural Machine Translation Olof Mogren 753 tutorial 103.91
90 MACHINE LEARNING WITH MISSING LABELS PART 3: EXPERIMENTS Charles H. Martin 581 tutorial 103.87
91 40 Interview Questions asked at Startups in Machine Learning / Data Science ANALYTICS VIDHYA CONTENT TEAM 107 tutorial 103.60
92 Generative Models Andrej Karpathy, Pieter Abbeel, Greg Brockman, Peter Chen, Vicki Cheung, Rocky Duan, Ian Goodfellow, Durk Kingma, Jonathan Ho, Rein Houthooft, Tim Salimans, John Schulman, Ilya Sutskever, Wojciech Zar 756 resource 102.97
93 Deep Learning 2: Part 2 Lesson 12 Hiromi Suenaga 711 resource 102.75