View Project


Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network


Cognitive NLP systems - i.e. , NLP systems that make use of behavioral data - augment traditional text-based features with cognitive features extracted from eye-movement patterns, EEG signals, brain-imaging etc.. Such extraction of features is typically manual. We contend that manual extraction of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection , and that even the extraction and choice of features should be delegated to the learning system. We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of human readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sarcasm detection. Our proposed framework is based on Convolutional Neural Network (CNN). The CNN learns features from both gaze and text and uses them to classify the input text. We test our technique on published sentiment and sarcasm labeled datasets, enriched with gaze information, to show that using a combination of automatically learned text and gaze features often yields better classification performance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic
1 Deep Learning for Sentiment Analysis : {A} Survey Lei Zhang and Shuai Wang and Bing Liu 1045
2 Deep Learning for Sentiment Analysis: A Survey Lei Zhang, Shuai Wang, Bing Liu 1122
3 A Survey on Deep Learning based Brain Computer Interface: Recent Advances and New Frontiers Xiang Zhang 1183
4 A Survey on Deep Learning in Medical Image Analysis "Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindr... 1193
5 Computational Analysis of Affect and Emotion in Language Saif M. Mohammad, Cecilia Ovesdotter Alm 1122
6 A Year In Computer Vision Benjamin F. Duffy, Daniel R. Flynn 862
7 Recent Trends in Deep Learning Based Natural Language Processing Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria 1181
8 Affect Detection from Text – from Affect Sciences to Computational Models Alexandra Balahur 1195
9 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
10 A Survey of Word Embeddings Evaluation Methods Amir Bakarov 1186
11 403 Forbidden None
12 Using human brain activity to guide machine learning Ruth Fong, Walter Scheirer, David Cox 9999
13 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 1129
14 A Survey on Deep Learning for Named Entity Recognition Li, Jing and Sun, Aixin and Han, Jianglei and Li, Chenliang 1089
15 Evaluative language beyond bags of words: Linguistic insights and computational applications Benamara, Farah and Taboada, Maite and Mathieu, Yannick 1265
16 Neural Information Retrieval: A Literature Review Ye Zhang, Md Mustafizur Rahman, Alex Braylan, Brandon Dang, Heng-Lu... 1169
17 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizu... 1183
18 57 Summaries of Machine Learning and NLP Research Marek Rei 1201
19 Opinion mining and sentiment analysis Bo Pang and Lillian Lee 1122
20 Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation Albert Gatt, Emiel Krahmer 1136
21 On the Origin of Deep Learning Haohan Wang, Bhiksha Raj 711
22 On the Origin of Deep Learning Haohan Wang and Bhiksha Raj and Eric P. Xing 1183
23 Akshat Maheshwari Ori Cohen 1181
24 A Review of 40 Years of Cognitive Architecture Research: Focus on Perception, Attention, Learning and Applications Iuliia Kotseruba, Oscar J Avella Gonzalez, John K Tsotsos 9999
25 A Survey of the Usages of Deep Learning in Natural Language Processing Daniel W. Otter, Julian R. Medina, Jugal K. Kalita 1181
26 De-identification of Patient Notes with Recurrent Neural Networks Franck Dernoncourt, Ji Young Lee, Ozlem Uzuner, Peter Szolovits 9999
27 Knowledge Base Population using Semantic Label Propagation Lucas Sterckx, Thomas Demeester, Johannes Deleu, Chris Develder 9999
28 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
29 Natural Language Processing for Information Extraction Sonit Singh 1089
30 Natural language based financial forecasting: a survey Frank Z. Xing, Erik Cambria, Roy E. Welsch 9999
31 Relation Extraction: A Survey Sachin Pawara, Girish K. Palshikara, Pushpak Bhattacharyya 1090
32 Recent Advances in Convolutional Neural Networks Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy,... 9999
33 Multilingual Sentiment and Subjectivity Analysis Rada Mihalcea, Carmen Banea, Janyce Wiebe 1122
34 Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing Edoardo Maria Ponti, Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi... 1083
35 Sentiment Analysis and Opinion Mining Bing Liu 1122
36 A Survey on Open Information Extraction Christina Niklaus, Matthias Cetto, Andre Freitas, and Siegfried Han... 1169
37 The 7 NLP Techniques That Will Change How You Communicate in the Future (Part II) James Le 1065
38 The Handbook of Computational Linguistics and Natural Language Processing "Alexander Clark, Chris Fox, and Shalom Lappin" 1557
39 Sentiment Analysis Subhabrata Mukherjee 1122
40 Fine-grained Sentiment Analysis in Python (Part 1) - Towards Data Science Not Available None
41 The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan B... 9999
42 Semantic Parsing 2, Question Answering Mohit Bansal 1125
43 Applications of Social Media Text Analysis Atefeh Farzindar, Diana Inkpen 1231
44 Sentiment Analysis and Opinion Mining Bing Liu 1122
45 Deep Learning for NLP, advancements and trends in 2017 Javier 1181
46 Negated bio-events: analysis and identification Raheel Nawaz, Paul Thompson, Sophia Ananiadou 9999
47 Analysis Methods in Neural Language Processing: A Survey Yonatan Belinkov and James Glass 1083
48 Mathematics of Deep Learning Raja Giryes, René Vidal 1183
49 Neural Network Methods for Natural Language Processing Yoav Goldberg 1183
50 Deep Learning applied to Image and Text matching Afroze Ibrahim Baqapuri 9999