View Project


Obtaining referential word meanings from visual and distributional information: Experiments on object naming


We investigate object naming, which is an important sub-task of referring expression generation on real-world images. As opposed to mutually exclusive labels used in object recognition, object names are more flexible, subject to communicative preferences and semantically related to each other. Therefore, we investigate models of referential word meaning that link visual to lexical information which we assume to be given through distributional word embeddings. We present a model that learns individual predictors for object names that link visual and distributional aspects of word meaning during training. We show that this is particularly beneficial for zero-shot learning, as compared to projecting visual objects directly into the distributional space. In a standard object naming task, we find that different ways of combining lexical and visual information achieve very similar performance, though experiments on model combination suggest that they capture complementary aspects of referential meaning.


Login to edit or delete this resource.

Suggested Topics (up to Top 50)

Full Matches (full topic name in abstract)

Partial Matches (at least half of words topic name appear in abstract)

Suggested Resources

Uses abstract to search the content of resources available in Topics. Sorted by relevance.

# Title Author Topic
1 Towards Multi-Agent Communication-Based Language Learning Angeliki Lazaridou, Nghia The Pham, Marco Baroni 9999
2 From visual attributes to adjectives through decompositional distributional semantics Angeliki Lazaridou, Georgiana Dinu, Adam Liska, Marco Baroni 9999
3 Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation Albert Gatt, Emiel Krahmer 1136
4 The Handbook of Computational Linguistics and Natural Language Processing "Alexander Clark, Chris Fox, and Shalom Lappin" 1557
5 Computational Modeling of Human Language Acquisition Afra Alishahi 1258
6 Attributes as Semantic Units between Natural Language and Visual Recognition Marcus Rohrbach 9999
7 HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment Ivan Vulic?, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen 9999
8 A Survey on Multi-output Learning "Donna Xu, Yaxin Shi, Ivor W. Tsang, Yew-Soon Ong, Chen Gong, and X... 1563
9 Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation Angeliki Lazaridou, Dat Tien Nguyen, Raffaella Bernardi, Marco Baroni 9999
10 Multiword Expression Processing: A Survey Mathieu Constant, Gül?en Eryi?it, Johanna Monti, Lonneke van der Plas 1122
11 100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask Emily M. Bender, Alex Lascarides 1120
12 Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erk... 9999
13 We Are Humor Beings: Understanding and Predicting Visual Humor Arjun Chandrasekaran, Ashwin K. Vijayakumar, Stanislaw Antol, Mohit... 9999
14 Hierarchy of Scales in Language Dynamics Richard A Blythe 9999
15 Multiword Expression Processing: A Survey Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Pl... 1254
16 Multiword expression processing: A survey Constant, Mathieu and Eryi{\u{g}}it, G{\ü}l{\c{s}}en and Monti, Joh... 1556
17 Modern Deep Learning Techniques Applied to Natural Language Processing Elvis Saravia, Soujanya Poria 1183
18 Vector Space Models of Lexical Meaning Stephen Clark 1186
19 Recursive neural networks can learn logical semantics Samuel R Bowman, Christopher Potts, Christopher D Manning 9999
20 Speech and Language Processing "Daniel Jurafsky, James H. Martin" 1252
21 Evaluative language beyond bags of words: Linguistic insights and computational applications Benamara, Farah and Taboada, Maite and Mathieu, Yannick 1265
22 Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing Edoardo Maria Ponti, Helen O'Horan, Yevgeni Berzak, Ivan Vulić, Roi... 1083
23 Introduction to NLP and Deep Learning Richard Socher 1045
24 A Survey of Word Embeddings Evaluation Methods Amir Bakarov 1186
25 Recent Trends in Deep Learning Based Natural Language Processing Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria 1181
26 Encode, Review, and Decode: Reviewer Module for Caption Generation Zhilin Yang, Ye Yuan, Yuexin Wu, Ruslan Salakhutdinov, William W Cohen 9999
27 Deep Learning for Natural Language Processing: Introductioin Richard Socher 1009
28 Transfer Learning - Machine Learnings Next Frontier Sebastian Ruder 1323
29 Transfer Learning - Machine Learning's Next Frontier Sebastian Ruder 9999
30 Deep Learning applied to Image and Text matching Afroze Ibrahim Baqapuri 9999
31 Akshat Maheshwari Ori Cohen 1181
32 Learning Semantically and Additively Compositional Distributional Representations Ran Tian, Naoaki Okazaki, Kentaro Inui 9999
33 Neural Models for Information Retrieval Bhaskar Mitra, Nick Craswell 1089
34 Graph Representation Learning Jure Leskovec 1257
35 Language+Vision Mohit Bansal 1338
36 Captioning Images with Diverse Objects Subhashini Venugopalan, Lisa Anne Hendricks, Marcus Rohrbach, Raymo... 9999
37 Semantic Similarity Frontiers: From Concepts to Documents David Jurgens, Mohammad Taher Pilehvar 1111
38 Neural Information Retrieval: At the End of the Early Years Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizu... 1183
39 57 Summaries of Machine Learning and NLP Research Marek Rei 1201
40 From Frequency to Meaning: Vector Space Models of Semantics Peter D. Turney, Patrick Pantel 1115
41 Part-of-Speech Tagging, NER, Sequence Labeling, Coreference Resolution Mohit Bansal 1085
42 A Survey of Paraphrasing and Textual Entailment Methods "Ion Androutsopoulos, Prodromos Malakasiotis" 1117
43 A Year In Computer Vision Benjamin F. Duffy, Daniel R. Flynn 862
44 Deep Learning in NLP Vered Shwartz 1181
45 Symbolic Reasoning (Symbolic AI) and Machine Learning | Skymind Not Available None
46 A Review of 40 Years of Cognitive Architecture Research: Focus on Perception, Attention, Learning and Applications Iuliia Kotseruba, Oscar J Avella Gonzalez, John K Tsotsos 9999
47 Neural Information Retrieval: A Literature Review Ye Zhang, Md Mustafizur Rahman, Alex Braylan, Brandon Dang, Heng-Lu... 1169
48 Analysis Methods in Neural Language Processing: A Survey Yonatan Belinkov and James Glass 1083
49 Representing Meaning with a Combination of Logical Form and Vectors Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, Raymond... 9999
50 A Survey of the Usages of Deep Learning in Natural Language Processing Daniel W. Otter, Julian R. Medina, Jugal K. Kalita 1181