4 papers · 1 filter
Aligned but Not Partner-Specific: How Multimodal LLM Agents Succeed in Reference Games Without Forming Conceptual Pacts
Po-Ya Angela Wang, Chinmaya Mishra, Aslı Özyürek +2
Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific expressions grounded in shared interaction history; tha…
Using Perspectival Words Is Harder Than Vocabulary Words for Humans and Even More So for Multimodal Language Models
Dota Tianai Dong, Yifan Luo, Po-Ya Angela Wang +2
Multimodal language models (MLMs) increasingly demonstrate human-like communication, yet their use of everyday perspectival words remains poorly understood. To address this gap, we…
Lexical Retrieval Hypothesis in Multimodal Context
Po-Ya Angela Wang, Pin-Er Chen, Hsin-Yu Chou +2
Multimodal corpora have become an essential language resource for language science and grounded natural language processing (NLP) systems due to the growing need to understand and…
Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis
Pin-Er Chen, Po-Ya Angela Wang, Hsin-Yu Chou +2
This paper explores the grounding issue regarding multimodal semantic representation from a computational cognitive-linguistic view. We annotate images from the Flickr30k dataset w…