View Project


Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network


Cognitive NLP systems - i.e. , NLP systems that make use of behavioral data - augment traditional text-based features with cognitive features extracted from eye-movement patterns, EEG signals, brain-imaging etc.. Such extraction of features is typically manual. We contend that manual extraction of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection , and that even the extraction and choice of features should be delegated to the learning system. We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of human readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sarcasm detection. Our proposed framework is based on Convolutional Neural Network (CNN). The CNN learns features from both gaze and text and uses them to classify the input text. We test our technique on published sentiment and sarcasm labeled datasets, enriched with gaze information, to show that using a combination of automatically learned text and gaze features often yields better classification performance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features.



Suggested Topics

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek K 713 resource 249.66
2 Codra: A Novel Discriminative Framework for Rhetorical Analysis Shafiq Joty, Giuseppe Carenini, Raymond T. Ng 999 paper 242.12
3 Highlights of EMNLP 2017: Exciting Datasets, Return of the Clusters, and More! Sebastian Ruder 641 resource 224.55
4 Recent Advances in Document Summarization Jin-ge Yao, Xiaojun Wan, Jianguo Xiao 421 survey 220.41
5 Deep Learning for NLP, advancements and trends in 2017 Javier 711 resource 217.82
6 A Comparative Analysis of ChatBots APIs Author Unknown 921 resource 209.73
7 Discriminative Syntax-based Word Ordering for Text Generation Yue Zhang, Stephen Clark 999 paper 208.79
8 Negated bio-events: analysis and identification Raheel Nawaz, Paul Thompson, Sophia Ananiadou 999 paper 207.70
9 A Gentle Introduction to Machine Learning Author Unknown 711 tutorial 207.50
10 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 978 tutorial 203.34
11 Understanding Convolutional Neural Networks for NLP Denny Britz 744 tutorial 200.63
12 The Definitive Guide to Natural Language Processing Javier Couto 133 tutorial 198.88
13 Tombones Computer Vision Blog Tomasz Malisiewicz 958 resource 196.39
14 Introduction to Natural Language Processing (NLP) 2016 Matt Kiser 133 tutorial 195.23
15 Similarity-driven Semantic Role Induction via Graph Partitioning Joel Lang, Mirella Lapata 999 paper 194.78
16 Machine Learning for Humans Vishal Maini, Samer Sabri 134 tutorial 192.57
17 10 Applications of Artificial Neural Networks in Natural Language Processing Olga Davydova 811 resource 191.75
18 The Neural Network Zoo Fjodor Van Veen 712 tutorial 191.64
19 An Overview of Multi-Task Learning in Deep Neural Networks Sebastian Ruder 829 tutorial 191.28
20 A survey of transfer learning Karl Weiss, Taghi M. Khoshgoftaar and DingDing Wang 978 resource 190.75
21 Using Artificial Intelligence to Augment Human Intelligence Shan Carter, Michael Nielsen 811 resource 190.65
22 Clustering cliques for graph-based summarization of the biomedical research literature Han Zhang, Marcelo Fiszman, Dongwook Shin, Bartomiej Wilkowski, Thomas Rindflesch 999 paper 189.77
23 Introduction to Visual Question Answering: Datasets, Approaches and Evaluation Javier Couto 411 resource 186.71
24 Recurrent Neural Networks Stephen Grossberg 741 paper 185.36
25 Learning AI if You Suck at Math?—?P5?—?Deep Learning and Convolutional Neural Nets in Plain English! Daniel Jeffries 811 tutorial 182.35
26 DEEP LEARNING FOR CHATBOTS, PART 1 - INTRODUCTION Denny Britz 445 tutorial 182.09
27 An end to end implementation of a Machine Learning pipeline Spandan Madan 107 tutorial 181.84
28 30 Amazing Applications of Deep Learning Yaron Hadad 711 resource 180.75
30 Recurrent Neural Networks Tutorial, Part 1 - Introduction to RNNs Denny Britz 741 tutorial 180.12
31 Applied Deep Learning - Part 4: Convolutional Neural Networks Arden Dertat 744 resource 178.04
32 A Practitioner's Guide to Natural Language Processing (Part I)?—?Processing & Understanding Text Dipanjan (DJ) Sarker 112 resource 178.01
33 Gimli: open source and high-performance biomedical name recognition David Campos, Sergio Matos, Jose Oliveira 999 paper 176.48
34 Deep Learning for NLP Best Practices Sebastian Ruder 713 tutorial 176.21
35 Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience Ismael Rafols, Martin Meyer 999 paper 175.98
36 Rohan #2: Artificial intelligence, ?Progress/?Time Rohan Kapur 811 tutorial 175.30
37 A Microsoft CNTK tutorial in Python – build a neural network Andy Thomas 711 tutorial 175.28
38 Ideas on interpreting machine learning Patrick Hall, Wen Phan, SriSatish Ambati 134 tutorial 174.99
39 State-of-the-art neural coreference resolution for chatbots Thomas Wolf 756 tutorial 174.14
40 Learning AI if You Suck at Math?—?P7?—?The Magic of Natural Language Processing Daniel Jeffries 133 tutorial 174.08
41 Ultimate Guide to Understand & Implement Natural Language Processing (with codes in Python) Shivam Bansal 131 tutorial 173.93
42 Transfer Learning: Leverage Insights from Big Data Lars Hulstaert 753 resource 173.82
43 12 Frequently Asked Questions on Deep Learning (with their answers)! Analytics Vidhya Content Team 711 resource 171.43
44 Recurrent Neural Networks Tutorial, Part 2 - Implementing a RNN with Python, Numpy, and Theano Denny Britz 741 tutorial 170.69
45 RNNs in Tensorflow, a Practical Guide and Undocumented Features Denny Britz 741 tutorial 170.59
46 A survey of cross-lingual embedding models Sebastian Ruder 721 tutorial 169.87
47 The Unreasonable Effectiveness of Recurrent Neural Networks Andrej Karpathy 741 survey 169.46
48 Lexicalization and Generative Power in Ccg Marco Kuhlmann, Alexander Koller, Giorgio Satta 999 paper 168.25
49 Text Classification with TensorFlow Estimators Julian Eisenschlos, Sebastian Ruder 731 resource 167.72
50 An Intuitive Explanation of Convolutional Neural Networks Ujjwal Karn 744 tutorial 167.62
51 ResNet, AlexNet, VGG, Inception: Understanding various architectures of Convolutional Networks Koustubh 744 resource 167.48
52 Visualizing Representations: Deep Learning and Human Beings Christopher Olah 713 tutorial 167.48
53 Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano Denny Britz 742 tutorial 166.88
54 Twitter Sentiment Analysis Using Combined LSTM-CNN Models Author Unknown 381 resource 166.31
55 Multi-Task Learning Objectives for Natural Language Processing Author Unknown 133 resource 165.07
56 The Building Blocks of Interpretability Chris Olah 614 resource 164.92
57 Natural Language Processing (NLP) for Computational Social Science Cristian Danescu-Niculescu-Mizil, Lillian Lee 133 tutorial 164.78
58 An Intuitive Guide to Linear Algebra Kalid Azad 121 tutorial 163.19
59 Rohan & Lenny #3: Recurrent Neural Networks & LSTMs Rohan Kapur 741 tutorial 163.03
60 Introduction to Word2Vec Skymind 721 tutorial 162.93
61 Learning to Segment Piotr Dollar 862 tutorial 162.48
62 Deep Learning 2: Part 1 Lesson 4 Hiromi Suenaga 711 resource 162.46
63 Introduction to Learning to Trade with Reinforcement Learning Denny Britz 857 resource 161.98
64 Architecture of Convolutional Neural Networks (CNNs) demystified Dishashree Gupta 744 tutorial 161.93
65 A Beginners Guide to Deep Learning Kumar Shridhar 711 tutorial 161.88
66 A curated list of data science, analysis and visualization tools QuantMind 134 resource 161.16
67 Text Classifier Algorithms in Machine Learning Roman Trusov 542 tutorial 161.01
68 Summaries and notes on Deep Learning research papers Denny Britz 713 resource 160.89
69 Natural Language Processing for Beginners: Using TextBlob Shubham Jain 731 resource 160.83
70 An Overview of Proxy-label Approaches for Semi-supervised Learning Sebastian Ruder 581 resource 160.53
71 From Natural Language Processing to Ar4ficial Intelligence Jonathan Mugan 133 tutorial 160.48
72 What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders? ffriend 744 tutorial 160.13
73 Making computers explain themselves Larry Hardesty 713 resource 159.73
74 New approaches to Deep Networks:Capsules (Hinton), HTM (Numenta), Sparsey (Neurithmic Systems) and RCN (Vicarious) Gideon Kowadlo 731 resource 159.72
75 Prodigy: A new tool for radically efficient machine teaching Matthew Honnibal, Ines Montani 134 resource 159.38
76 Machine Learning Glossary Author Unknown 107 resource 159.05
77 Simple, Strong Deep-Learning Baselines for NLP in several frameworks Dan Pressel 713 library 158.88
78 What is machine learning? Everything you need to know Nick Heath 711 resource 158.84
79 Getting Started with Sentiment Analysis bogdani 381 resource 158.80
80 TensorFlow RNN Tutorial Matt Mollison 731 resource 158.49
81 Neural Text Embeddings for IR Bhaskar Mitra, Nick Craswell 721 tutorial 158.20
82 Comprehensive Guide on t-SNE algorithm with implementation in R & Python SAURABH.JAJU2 341 tutorial 158.15
83 Keras and Convolutional Neural Networks (CNNs) Adrian Rosebrock 744 resource 157.84
84 A Beginner's guide to Recurrent Networks and LSTMs Skymind 742 tutorial 157.82
85 A social network's changing statistical properties and the quality of human innovation Brian Uzzi 999 paper 157.62
86 Rohan & Lenny #2: Convolutional Neural Networks Lenny Khazan 744 tutorial 157.54
87 Transfer Learning Niklas Donges 978 resource 157.37
88 Machine Learning Basics: a Guide for the Perplexed Will Gannon 134 resource 157.36
89 A Gentle Introduction to Neural Networks for Machine Learning James Le 711 resource 156.94
90 Uncovering the Intuition behind Capsule Networks and Inverse Graphics: Part I Tanay Kothari 641 resource 156.83
91 Uncovering the Intuition behind Capsule Networks and Inverse Graphics Tanay Kothari 711 resource 156.80