View Project


Obtaining referential word meanings from visual and distributional information: Experiments on object naming


We investigate object naming, which is an important sub-task of referring expression generation on real-world images. As opposed to mutually exclusive labels used in object recognition, object names are more flexible, subject to communicative preferences and semantically related to each other. Therefore, we investigate models of referential word meaning that link visual to lexical information which we assume to be given through distributional word embeddings. We present a model that learns individual predictors for object names that link visual and distributional aspects of word meaning during training. We show that this is particularly beneficial for zero-shot learning, as compared to projecting visual objects directly into the distributional space. In a standard object naming task, we find that different ways of combining lexical and visual information achieve very similar performance, though experiments on model combination suggest that they capture complementary aspects of referential meaning.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Attributes as Semantic Units between Natural Language and Visual Recognition Marcus Rohrbach 9999
2 Computational Modeling of Human Language Acquisition Afra Alishahi 1258
3 Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation Angeliki Lazaridou, Dat Tien Nguyen, Raffaella Bernardi, Marco Baroni 9999
4 We Are Humor Beings: Understanding and Predicting Visual Humor Arjun Chandrasekaran, Ashwin K. Vijayakumar, Stanislaw Antol, Mohit... 9999
5 Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erk... 9999
6 Hierarchy of Scales in Language Dynamics Richard A Blythe 9999
7 Vector Space Models of Lexical Meaning Stephen Clark 1186
8 Multiword Expression Processing: A Survey Mathieu Constant, Gül?en Eryi?it, Johanna Monti, Lonneke van der Plas 1122
9 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
10 100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask Emily M. Bender, Alex Lascarides 1120
11 Multiword Expression Processing: A Survey Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Pl... 1254
12 Multiword expression processing: A survey Constant, Mathieu and Eryi{\u{g}}it, G{\ü}l{\c{s}}en and Monti, Joh... 1556
13 Introduction to NLP and Deep Learning Richard Socher 1045
14 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 1323
15 Transfer Learning - Machine Learning's Next Frontier Sebastian Ruder 9999
16 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
17 Evaluative language beyond bags of words: Linguistic insights and computational applications Benamara, Farah and Taboada, Maite and Mathieu, Yannick 1265
18 Deep Learning for Natural Language Processing: Introductioin Richard Socher 1009
19 Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing Edoardo Maria Ponti, Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi... 1083
20 57 Summaries of Machine Learning and NLP Research Marek Rei 1201
21 Graph Representation Learning Jure Leskovec 1257
22 Language+Vision Mohit Bansal 1338
23 Semantic Similarity Frontiers: From Concepts to Documents David Jurgens, Mohammad Taher Pilehvar 1111
24 Deep Learning in NLP Vered Shwartz 1181
25 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizu... 1183
26 A Year In Computer Vision Benjamin F. Duffy, Daniel R. Flynn 862
27 Part-of-Speech Tagging, NER, Sequence Labeling, Coreference Resolution Mohit Bansal 1085
28 Representing Meaning with a Combination of Logical Form and Vectors Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, Raymond... 9999
29 Representation Learning with Networks Jure Leskovec 1180
30 Naive Bayes and Sentiment Classification Daniel Jurafsky, James H. Martin 1122
31 Unsupervised Compositionality Prediction of Nominal Compounds Cordeiro, Silvio and Villavicencio, Aline and Idiart, Marco and Ram... 1265
32 Embeddings for Word Sense Disambiguation: An Evaluation Study Ignacio Iacobacci, Mohammad Taher Pilehvar, Roberto Navigli 9999
33 Language+Robotics Mohit Bansal 1340
34 Domain Adaptation Algorithms for Natural Language Processing Qi Li 1323
35 Anaphora With Non-nominal Antecedents in Computational Linguistics: a Survey Varada Kolhatkar, Adam Roussel, Stefanie Dipper, Heike Zinsmeister 1090
36 Anaphora with non-nominal antecedents in computational linguistics: A survey Kolhatkar, Varada and Roussel, Adam and Dipper, Stefanie and Zinsme... 1556
37 Deep Learning for Sentiment Analysis : {A} Survey Lei Zhang and Shuai Wang and Bing Liu 1045
38 Learning language through pictures Grzegorz Chrupa?a, Ákos Kádár, Afra Alishahi 9999
39 Learning Semantic Relations from Text Preslav Nakov, Diarmuid O' Seaghdha, Vivi Nastase, Stan Szpakowicz 1119
40 Deep Learning for NLP: An Overview of Recent Trends Elvis 1181
41 Neural Vector Spaces for Unsupervised Information Retrieval Christophe Van Gysel, Maarten De Rijke, Evangelos Kanoulas 1169
42 Generalized Language Models None
43 A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment Haonan Yu, Haichao Zhang, Wei Xu 9999
44 Deep Compositional Question Answering with Neural Module Networks Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein 9999
45 Generative Adversarial Networks (GANs) Ian Goodfellow 1116
46 Generative Adversarial Networks (GANs) Ian Goodfellow 1200
47 Modeling Newswire Events using Neural Networks for Anomaly Detection Pradeep Dasigi, Eduard Hovy 9999
48 Distributional Measures of Semantic Distance: A Survey Saif M. Mohammad, Graeme Hirst 9999
49 On sentence representations, pt. 1: what can you fit into a single #$!%@*&% blog post? N/A 1188
50 Topics, Trends, and Resources in NLP Mohit Bansal 1065