View Project


Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network


Cognitive NLP systems - i.e. , NLP systems that make use of behavioral data - augment traditional text-based features with cognitive features extracted from eye-movement patterns, EEG signals, brain-imaging etc.. Such extraction of features is typically manual. We contend that manual extraction of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection , and that even the extraction and choice of features should be delegated to the learning system. We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of human readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sarcasm detection. Our proposed framework is based on Convolutional Neural Network (CNN). The CNN learns features from both gaze and text and uses them to classify the input text. We test our technique on published sentiment and sarcasm labeled datasets, enriched with gaze information, to show that using a combination of automatically learned text and gaze features often yields better classification performance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Deep Learning for Sentiment Analysis : {A} Survey Lei Zhang and Shuai Wang and Bing Liu 1045
2 Computational Analysis of Affect and Emotion in Language Saif M. Mohammad, Cecilia Ovesdotter Alm 1122
3 A Year In Computer Vision Benjamin F. Duffy, Daniel R. Flynn 862
4 Affect Detection from Text – from Affect Sciences to Computational Models Alexandra Balahur 1195
5 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
6 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 1129
7 A Survey on Deep Learning for Named Entity Recognition Li, Jing and Sun, Aixin and Han, Jianglei and Li, Chenliang 1089
8 Evaluative language beyond bags of words: Linguistic insights and computational applications Benamara, Farah and Taboada, Maite and Mathieu, Yannick 1265
9 57 Summaries of Machine Learning and NLP Research Marek Rei 1201
10 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizu... 1183
11 Opinion mining and sentiment analysis Bo Pang and Lillian Lee 1122
12 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
13 Knowledge Base Population using Semantic Label Propagation Lucas Sterckx, Thomas Demeester, Johannes Deleu, Chris Develder 9999
14 Multilingual Sentiment and Subjectivity Analysis Rada Mihalcea, Carmen Banea, Janyce Wiebe 1122
15 The 7 NLP Techniques That Will Change How You Communicate in the Future (Part II) James Le 1065
16 Deep Learning for NLP, advancements and trends in 2017 Javier 1181
17 Applications of Social Media Text Analysis Atefeh Farzindar, Diana Inkpen 1231
18 Sentiment Analysis and Opinion Mining Bing Liu 1122
19 Natural language based financial forecasting: a survey Frank Z. Xing, Erik Cambria, Roy E. Welsch 9999
20 Negated bio-events: analysis and identification Raheel Nawaz, Paul Thompson, Sophia Ananiadou 9999
21 Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing Edoardo Maria Ponti, Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi... 1083
22 Deep Learning for NLP: An Overview of Recent Trends Elvis 1181
23 Sentiment Analysis and Opinion Mining Bing Liu 1122
24 Mathematics of Deep Learning Raja Giryes, René Vidal 1183
25 Semantic Parsing 2, Question Answering Mohit Bansal 1125
26 Opinion Mining: Exploiting the Sentiment of the Crowd Diana Maynard, Adam Funk, Kalina Bontcheva 1122
27 Neural Network Methods for Natural Language Processing Yoav Goldberg 1183
28 On sentence representations, pt. 1: what can you fit into a single #$!%@*&% blog post? N/A 1188
29 Sentiment Analysis in Practice Yongzheng (Tiger) Zhang, Dan Shen, Catherine Baudin 1122
30 Recent Advances in Recurrent Neural Networks Hojjat Salehinejad and Julianne Baarbe and Sharan Sankar and Joseph... 1190
31 Computational Modeling of Human Language Acquisition Afra Alishahi 1258
32 Jumping NLP Curves: A Review of Natural Language Processing Research Erik Cambria, Bebo White 1053
33 Multiword Expression Processing: A Survey Mathieu Constant, Gül?en Eryi?it, Johanna Monti, Lonneke van der Plas 1122
34 Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erk... 9999
35 Improving sentence compression by learning to predict gaze Sigrid Klerke, Yoav Goldberg, Anders Søgaard 1134
36 Scholarly Data Mining: Making Sense of Scientific Literature Horacio Saggion & Francesco Ronzano 1151
37 Natural Language Processing for Intelligent Access to Scientific Information Francesco Ronzano, Horacio Saggion 1089
38 Language+Vision Mohit Bansal 1338
39 10 Applications of Artificial Neural Networks in Natural Language Processing Olga Davydova 1300
40 Improving sentence compression by learning to predict gaze Sigrid Klerke, Yoav Goldberg, Anders Søgaard 1134
41 NLP's ImageNet moment has arrived Sebastian Ruder 1338
42 Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text Sahil Garg, Aram Galstyan, Ulf Hermjakob, Daniel Marcu 9999
43 Topics, Trends, and Resources in NLP Mohit Bansal 1065
44 Sentiment Analysis and Subjectivity Bing Liu 1122
45 Convolutional Neural Networks: The Biologically-Inspired Model James Le 1193
46 Information Extraction Katharina Kaiser and Silvia Miksch 1089
47 Deep Learning in Neural Networks: An Overview J{\"{u}}rgen Schmidhuber 1006
48 Deep Learning for Natural Language Processing Tianchuan Du, Vijay K. Shanker 1181
49 Scalable Construction and Reasoning of Massive Knowledge Bases Xiang Ren, Nanyun Peng, William Yang Wang 1216
50 Scalable Construction and Reasoning of Massive Knowledge Bases Xiang Ren, Nanyun Peng, William Yang Wang 1216