14 papers
T-WLA: Bridging Slow Vision and Fast Touch
Kaichen Zhou, Yuzhen Chen, Hannah Yao +14
In contact-rich manipulation, contact conditions can change before a visual policy updates its action plan. Recomputing the plan through the visual policy adds latency, while tacti…
Representation-guided in-context learning for medical image interpretation with multimodal large language models
Minda Zhao, Fangyu Hu, Yan Luo +8
Medical image interpretation is central to diagnosis and care, yet adapting general-purpose multimodal large language models (MLLMs) often requires resource-intensive domain-specif…
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
Haoyu Chen, Kaichen Zhou, Hang Hua +11
Video generation models aspire to simulate dynamic environments, and several benchmarks now evaluate memory consistency across frames. However, most assess consistency only while t…
EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
Yue Ma, Xu Ye, Qinghe Wang +9
Generating high-fidelity visual effects (VFX) typically demands massive datasets and prohibitive computational power due to the intricate coupling of spatial textures and temporal…
PaintCopilot: Modeling Painting as Autonomous Artistic Continuation
Yunge Wen, Yaluo Wang, Yuancheng Shen +2
Existing neural painting methods are target-driven: given a reference image, strokes are optimized to reconstruct it, fixing the outcome before painting begins. We instead ask whet…
GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
Kaichen Zhou, Yuzhen Chen, Fangneng Zhan +8
Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the genera…
Stream3D: Sequential Multi-View 3D Generation via Evidential Memory
Kaichen Zhou, Zeyang Bai, Xinhai Chang +3
View-conditioned 3D generators such as SAM 3D, TRELLIS, and Hunyuan3D produce high-quality object reconstructions from a single view, but real-world visual observation often arrive…
DENALI: A Dataset Enabling Non-Line-of-Sight Spatial Reasoning with Low-Cost LiDARs
Nikhil Behari, Diego Rivero, Luke Apostolides +3
Consumer LiDARs in mobile devices and robots typically output a single depth value per pixel. Yet internally, they record full time-resolved histograms containing direct and multi-…
Abstract 3D Perception for Spatial Intelligence in Vision-Language Models
Yifan Liu, Fangneng Zhan, Kaichen Zhou +3
Vision-language models (VLMs) struggle with 3D-related tasks such as spatial cognition and physical understanding, which are crucial for real-world applications like robotics and e…
PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perception
Kaichen Zhou, Yuhan Wang, Grace Chen +5
Recent 3D feed-forward models, such as the Visual Geometry Grounded Transformer (VGGT), have shown strong capability in inferring 3D attributes of static scenes. However, since the…
Massively Multimodal Foundation Models: A Framework for Capturing Interactions with Specialized Mixture-of-Experts
Xing Han, Hsing-Huan Chung, Joydeep Ghosh +2
Modern applications increasingly involve many heterogeneous input streams, such as clinical sensors, wearable device data, imaging, and text, each with distinct measurement models,…
Sotopia-RL: Reward Design for Social Intelligence
Haofei Yu, Zhengyang Qi, Yining Zhao +6
Social intelligence has become a critical capability for large language models (LLMs), enabling them to engage effectively in real-world social tasks such as collaboration and nego…
MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping
Xiaojun Shan, Qi Cao, Xing Han +2
Recent advances in multimodal foundation models have achieved state-of-the-art performance across a range of tasks. These breakthroughs are largely driven by new pre-training parad…
Visual Acoustic Fields
Yuelei Li, Hyunjin Kim, Fangneng Zhan +7
Objects produce different sounds when hit, and humans can intuitively infer how an object might sound based on its appearance and material properties. Inspired by this intuition, w…