8 papers
The Mechanistic Emergence of Symbol Grounding in Language Models
Shuyu Wu, Ziqiao Ma, Xiaoxi Luo +4
Symbol grounding (Harnad, 1990) describes how symbols such as words acquire their meanings by connecting to real-world sensorimotor experiences. Recent work has shown preliminary e…
Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
Run Peng, Ziqiao Ma, Amy Pang +5
While Large Language Model (LLM) agents are often approached from the angle of action planning/generation to accomplish a goal (e.g., given by language descriptions), their abiliti…
AimBot: A Simple Auxiliary Visual Cue to Enhance Spatial Awareness of Visuomotor Policies
Yinpei Dai, Jayjun Lee, Yichi Zhang +6
In this paper, we propose AimBot, a lightweight visual augmentation technique that provides explicit spatial cues to improve visuomotor policy learning in robotic manipulation. Aim…
Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation
Ziqiao Ma, Jing Ding, Xuejun Zhang +6
Referring Expression Generation (REG) is a core task for evaluating the pragmatic competence of vision-language systems, requiring not only accurate semantic grounding but also adh…
Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutors
Jian Wang, Yinpei Dai, Yichi Zhang +3
Intelligent tutoring agents powered by large language models (LLMs) have been increasingly explored to deliver personalized knowledge in areas such as language learning and science…
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
Ziqiao Ma, Zekun Wang, Joyce Chai
Humans are efficient language learners and inherently social creatures. Our language development is largely shaped by our social interactions, for example, the demonstration and fe…