4 papers
Position: Reasoning After Perception Means Reasoning Without Vision
Hongcheng Gao, Zihao Huang, Jingyi Tang +12
A common belief in multimodal research is that the perceptual weaknesses of vision--language models can be compensated by stronger language reasoning (e.g., chain-of-thought, in-co…
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Hongcheng Gao, Hailong Qu, Jingyi Tang +18
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predomin…
Narrative Scaffolding: A Narrative-First Framework for Data-Driven Sensemaking
Oliver Huang, Muhammad Fatir, Steven Luo +3
When exploring data, analysts construct narratives about what the data means by asking questions, generating visualizations, reflecting on patterns, and revising their interpretati…
Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning
Majeed Kazemitabaar, Oliver Huang, Sangho Suh +2
Novice programmers are increasingly relying on Large Language Models (LLMs) to generate code for learning programming concepts. However, this interaction can lead to superficial en…