Objects of Unknown Categories
- Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories. Another strand of work in L&V aims at pragmatically informative rather than "correct" object descriptions, e.g. in reference games. We combine these lines of research and model zero-shot reference games, where a speaker needs to successfully refer to a novel object in an image. Inspired by models of "rational speech acts", we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories. As a result of this reasoning, the generator produces fewer nouns and names of distractor categories as compared to a literal speaker. We show that this conversational strategy for dealing with novel objects often improves communicative success, in terms of resolution accuracy of an automatic listener.
Author details: | Sina ZarriessGND, David SchlangenORCiDGND |
---|---|
ISBN: | 978-1-950737-48-2 |
Title of parent work (English): | The 57th Annual Meeting of the Association for Computational Linguistics |
Publisher: | Association for Computational Linguistics |
Place of publishing: | Stroudsburg |
Publication type: | Other |
Language: | English |
Year of first publication: | 2019 |
Publication year: | 2019 |
Release date: | 2021/06/22 |
Number of pages: | 6 |
First page: | 654 |
Last Page: | 659 |
Organizational units: | Humanwissenschaftliche Fakultät / Strukturbereich Kognitionswissenschaften / Department Linguistik |
DDC classification: | 4 Sprache / 40 Sprache / 400 Sprache |