View Project


TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension


We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions. We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers. We also present two baseline algorithms: a feature-based classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension. Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that TriviaQA is a challenging test bed that is worth significant future study.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 TriviaQA: A Large Scale Dataset for Reading Comprehension and Question Answering Mandar Joshi, Eunsol Choi, Daniel Weld, Luke Zettlemoyer 1126
2 awesome-qa seriousmac 1125
3 Question answering Sebastian Ruder 1031
4 Question answering | NLP-progress None
5 Standardized Tests as benchmarks for Artificial Intelligence? Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, Eric P. Xing 1300
6 SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang 9999
7 Recent Evolution of QA Datasets and Going Forward Jiwoong Im 1126
8 Semantic Parsing 2, Question Answering Mohit Bansal 1125
9 Neural Approaches to Conversational AI: Question Answering, Task-Oriented Dialogue and Chatbots: A Unified View Jianfeng Gao, Michel Galley, Lihong Li 1141
10 The Stanford Question Answering Dataset Pranav Rajpurkar 1125
11 Neural Approaches to Conversational {AI} Jianfeng Gao and Michel Galley and Lihong Li 1048
12 NLP’s generalization problem, and how researchers are tackling it Ana Marasovic 1181
13 Topics, Trends, and Resources in NLP Mohit Bansal 1065
14 Scalable Construction and Reasoning of Massive Knowledge Bases Xiang Ren, Nanyun Peng, William Yang Wang 1216
15 Scalable Construction and Reasoning of Massive Knowledge Bases Xiang Ren, Nanyun Peng, William Yang Wang 1216
16 Construction and Querying of Large-scale Knowledge Bases Xiang Ren, Yu Su, Xifeng Yan 1151
17 Key-Value Memory Networks for Directly Reading Documents Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Ant... 9999
18 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
19 Knowledge Base Population using Semantic Label Propagation Lucas Sterckx, Thomas Demeester, Johannes Deleu, Chris Develder 9999
20 WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia Daniel Hewlett, Alexandre Lacoste, Llion Jones, Illia Polosukhin, A... 9999
21 Language+Vision Mohit Bansal 1338
22 Paraphrase-Driven Learning for Open Question Answering Anthony Fader, Luke Zettlemoyer, Oren Etzioni 9999
23 Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erk... 9999
24 SRL, Semantic Parsing, Compositional Semantics 1 Mohit Bansal 1220
25 Learning to Compose Neural Networks for Question Answering Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein 9999
26 10 Exciting Ideas of 2018 in NLP Sebastian Ruder 1201
27 No time to read AI research? We summarized top 2018 papers for you Mariya Yao 1007
28 A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task Danqi Chen, Jason Bolton, Christopher D. Manning 9999
29 Attributes as Semantic Units between Natural Language and Visual Recognition Marcus Rohrbach 9999
30 Multilingual Relation Extraction using Compositional Universal Schema Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, Andrew... 9999
31 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 1129
32 Commonsense for Machine Intelligence: Text to Knowledge and Knowledge to Text Gerard de Melo, Niket Tandon and Aparna S. Varde 1216
33 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
34 Natural Language Inference, Reading Comprehension and Deep Learning Christopher Manning 1181
35 Large-scale Simple Question Answering with Memory Networks Antoine Bordes, Nicolas Usunier, Sumit Chopra, Jason Weston 9999
36 A large annotated corpus for learning natural language inference Samuel R Bowman, Gabor Angeli, Christopher Potts, Christopher D Man... 9999
37 Deep Compositional Question Answering with Neural Module Networks Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein 9999
38 Memory Networks for Language Understanding Jason Weston 1271
39 Memory Networks for Language Understanding Efstratios Gavves 1194
40 Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors Marco Baroni, Georgiana Dinu, Germán Kruszewski 9999
41 Deep Unordered Composition Rivals Syntactic Methods for Text Classification Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, Hal Daumé III 9999
42 Deep Unordered Composition Rivals Syntactic Methods for Text Classification Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, Hal Daum’e III 9999
43 Sentence Pair Scoring: Towards Unified Framework for Text Comprehension Petr Baudiš, Jan Šedivy? 9999
44 Learning Executable Semantic Parsers for Natural Language Understanding Percy Liang 9999
45 Words or Characters? Fine-grained Gating for Reading Comprehension Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W Cohen, R... 9999
46 Dialogue Learning With Human-In-The-Loop Jiwei Li, Alexander H Miller, Sumit Chopra, Marc'Aurelio Ranzato, J... 9999
47 Applications I: Reading comprehension / MemNets Greg Durrett 1194
48 Constructing and Mining Web-scale Knowledge Graphs Evgeniy Gabrilovich, Nicolas Usunier 1257
49 ACL 2018 Highlights: Understanding Representations and Evaluation in More Challenging Settings Sebastian Ruder 9999
50 A Survey on Deep Learning for Named Entity Recognition Li, Jing and Sun, Aixin and Han, Jianglei and Li, Chenliang 1089