View Project


Obtaining referential word meanings from visual and distributional information: Experiments on object naming


We investigate object naming, which is an important sub-task of referring expression generation on real-world images. As opposed to mutually exclusive labels used in object recognition, object names are more flexible, subject to communicative preferences and semantically related to each other. Therefore, we investigate models of referential word meaning that link visual to lexical information which we assume to be given through distributional word embeddings. We present a model that learns individual predictors for object names that link visual and distributional aspects of word meaning during training. We show that this is particularly beneficial for zero-shot learning, as compared to projecting visual objects directly into the distributional space. In a standard object naming task, we find that different ways of combining lexical and visual information achieve very similar performance, though experiments on model combination suggest that they capture complementary aspects of referential meaning.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic Medium Score
1 Towards Multi-Agent Communication-Based Language Learning Angeliki Lazaridou, Nghia The Pham, Marco Baroni 9999
2 From visual attributes to adjectives through decompositional distributional semantics Angeliki Lazaridou, Georgiana Dinu, Adam Liska, Marco Baroni 9999
3 Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation Albert Gatt, Emiel Krahmer 1136
4 Computational Modeling of Human Language Acquisition Afra Alishahi 1258
5 Attributes as Semantic Units between Natural Language and Visual Recognition Marcus Rohrbach 9999
6 Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation Angeliki Lazaridou, Dat Tien Nguyen, Raffaella Bernardi, Marco Baroni 9999
7 A Survey on Multi-output Learning "Donna Xu, Yaxin Shi, Ivor W. Tsang, Yew-Soon Ong, Chen Gong, and X... 1563
8 HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment Ivan Vulic?, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen 9999
9 100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask Emily M. Bender, Alex Lascarides 1120
10 Multiword Expression Processing: A Survey Mathieu Constant, Gül?en Eryi?it, Johanna Monti, Lonneke van der Plas 1122
11 We Are Humor Beings: Understanding and Predicting Visual Humor Arjun Chandrasekaran, Ashwin K. Vijayakumar, Stanislaw Antol, Mohit... 9999
12 Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erk... 9999
13 Hierarchy of Scales in Language Dynamics Richard A Blythe 9999
14 Vector Space Models of Lexical Meaning Stephen Clark 1186
15 Multiword Expression Processing: A Survey Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Pl... 1254
16 Multiword expression processing: A survey Constant, Mathieu and Eryi{\u{g}}it, G{\ü}l{\c{s}}en and Monti, Joh... 1556
17 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
18 A Technical Introduction to Statistical Natural Language Processing Jacob Eisenstein 1065
19 Recursive neural networks can learn logical semantics Samuel R Bowman, Christopher Potts, Christopher D Manning 9999
20 Evaluative language beyond bags of words: Linguistic insights and computational applications Benamara, Farah and Taboada, Maite and Mathieu, Yannick 1265
21 Speech and Language Processing "Daniel Jurafsky, James H. Martin" 1252
22 Introduction to NLP and Deep Learning Richard Socher 1045
23 Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing Edoardo Maria Ponti, Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi... 1083
24 A Survey of Text Classification Algorithms Charu C. Aggarwal, ChengXiang Zhai 542
25 A Survey of Word Embeddings Evaluation Methods Amir Bakarov 1186
26 Group Recommender Systems Judith Masthoff 9999
27 Deep Learning for Natural Language Processing: Introductioin Richard Socher 1009
28 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
29 Recent Trends in Deep Learning Based Natural Language Processing Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria 1181
30 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 1323
31 Transfer Learning - Machine Learning's Next Frontier Sebastian Ruder 9999
32 Learning Semantically and Additively Compositional Distributional Representations Ran Tian, Naoaki Okazaki, Kentaro Inui 9999
33 Deep Learning applied to Image and Text matching Afroze Ibrahim Baqapuri 9999
34 Graph Representation Learning Jure Leskovec 1257
35 Akshat Maheshwari Ori Cohen 1181
36 Neural Models for Information Retrieval Bhaskar Mitra, Nick Craswell 1089
37 Language+Vision Mohit Bansal 1338
38 Captioning Images with Diverse Objects Subhashini Venugopalan, Lisa Anne Hendricks, Marcus Rohrbach, Raymo... 9999
39 Semantic Similarity Frontiers: From Concepts to Documents David Jurgens, Mohammad Taher Pilehvar 1111
40 Successful Data Mining Methods for NLP Jiawei Han, Heng Ji, Yizhou Sun 1271
41 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizu... 1183
42 57 Summaries of Machine Learning and NLP Research Marek Rei 1201
43 From Frequency to Meaning: Vector Space Models of Semantics Peter D. Turney, Patrick Pantel 1115
44 Part-of-Speech Tagging, NER, Sequence Labeling, Coreference Resolution Mohit Bansal 1085
45 Deep Learning in NLP Vered Shwartz 1181
46 A Year In Computer Vision Benjamin F. Duffy, Daniel R. Flynn 862
47 Natural Language Processing with Python Steven Bird, Ewan Klein, Edward Loper 1064
48 Zero-Shot Learning Through Cross-Modal Transfer Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Chris... 9999
49 Representing Meaning with a Combination of Logical Form and Vectors Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, Raymond... 9999
50 Neural Information Retrieval: A Literature Review Ye Zhang, Md Mustafizur Rahman, Alex Braylan, Brandon Dang, Heng-Lu... 1169