19 citations · 29 across the 32 of their papers we have counts for
36 papers
SenseWeather: Multimodal Sensing for Robust Robotic Perception under Extreme Conditions
Fucai Ke, Sukai Huang, Junyi Hao +4
Robotic perception must remain reliable when extreme environmental conditions severely degrade the primary RGB observation. Although robotic platforms commonly carry complementary…
Feedforward Novel View Synthesis for Heterogeneous Cameras
Meng Wei, Cheng Zhang, Boying Li +4
Feed-forward novel view synthesis has recently shown promising results from sparse posed images, but most existing methods assume that context and target views share a fixed camera…
JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments
Zhixi Cai, Fucai Ke, Sukai Huang +4
In complex embodied visual reasoning scenarios, an agent often has only a limited field of view, and the evidence needed to answer a question may be distributed across time, viewpo…
Sample-Efficient Learning from Agent Experience
Chenhui Gou, Haoqin Tu, Yunhao Fang +2
Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offer…
ARIS: Agentic and Relationship Intelligence System for Social Robots
Stavya Datta, Fucai Ke, Leimin Tian +1
Foundational models have advanced social robotics, enabling richer perception and communicative interaction with users. However, current systems still struggle with multi-turn enga…
Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agents
Sukai Huang, Chenyuan Zhang, Fucai Ke +4
Instruction granularity is an important yet poorly controlled variable in language-guided embodied AI. Existing benchmarks typically pair each task with a single static instruction…
dinov3.seg: Open-Vocabulary Semantic Segmentation with DINOv3
Saikat Dutta, Biplab Banerjee, Hamid Rezatofighi
Open-Vocabulary Semantic Segmentation (OVSS) assigns pixel-level labels from an open set of text-defined categories, demanding reliable generalization to unseen classes at inferenc…
VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations
Fucai Ke, Zhixi Cai, Boying Li +6
Multi-view visual reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints, yet existing research has largely fo…
MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning
Zhixi Cai, Fucai Ke, Kevin Leo +4
Recent vision-language models have strong perceptual ability but their implicit reasoning is hard to explain and easily generates hallucinations on complex queries. Compositional m…
ASAP-Textured Gaussians: Enhancing Textured Gaussians with Adaptive Sampling and Anisotropic Parameterization
Meng Wei, Cheng Zhang, Jianmin Zheng +2
Recent advances have equipped 3D Gaussian Splatting with texture parameterizations to capture spatially varying attributes, improving the performance of both appearance modeling an…
Asset-Driven Sematic Reconstruction of Dynamic Scene with Multi-Human-Object Interactions
Sandika Biswas, Qianyi Wu, Biplab Banerjee +1
Real-world human-built environments are highly dynamic, involving multiple humans and their complex interactions with surrounding objects. While 3D geometry modeling of such scenes…
VQ-VA World: Towards High-Quality Visual Question-Visual Answering
Chenhui Gou, Zilong Chen, Zeyu Wang +10
This paper studies Visual Question-Visual Answering (VQ-VA): generating an image, rather than text, in response to a visual question -- an ability that has recently emerged in prop…
A Multi-Modal Neuro-Symbolic Approach for Spatial Reasoning-Based Visual Grounding in Robotics
Simindokht Jahangard, Mehrzad Mohammadi, Abhinav Dhall +1
Visual reasoning, particularly spatial reasoning, is a challenging cognitive task that requires understanding object relationships and their interactions within complex environment…
An Empirical Study on How Video-LLMs Answer Video Questions
Chenhui Gou, Ziyu Ma, Zicheng Duan +6
Taking advantage of large-scale data and pretrained language models, Video Large Language Models (Video-LLMs) have shown strong capabilities in answering video questions. However,…
JRDB-Reasoning: A Difficulty-Graded Benchmark for Visual Reasoning in Robotics
Simindokht Jahangard, Mehrzad Mohammadi, Yi Shen +2
Recent advances in Vision-Language Models (VLMs) and large language models (LLMs) have greatly enhanced visual reasoning, a key capability for embodied AI agents like robots. Howev…
Explain Before You Answer: A Survey on Compositional Visual Reasoning
Fucai Ke, Joy Hsu, Zhixi Cai +10
Compositional visual reasoning has emerged as a key research frontier in multimodal AI, aiming to endow machines with the human-like ability to decompose visual scenes, ground inte…
DPL: Decoupled Prototype Learning for Enhancing Robustness of Vision-Language Transformers to Missing Modalities
Jueqing Lu, Yuanyuan Qi, Xiaohao Yang +8
The performance of Visio-Language Transformers drops sharply when an input modality (e.g., image) is missing, because the model is forced to make predictions using incomplete infor…
Marginalized Generalized IoU (MGIoU): A Unified Objective Function for Optimizing Any Convex Parametric Shapes
Duy-Tho Le, Trung Pham, Jianfei Cai +1
Optimizing the similarity between parametric shapes is crucial for numerous computer vision tasks, where Intersection over Union (IoU) stands as the canonical measure. However, exi…
AerOSeg: Harnessing SAM for Open-Vocabulary Segmentation in Remote Sensing Images
Saikat Dutta, Akhil Vasim, Siddhant Gole +2
Image segmentation beyond predefined categories is a key challenge in remote sensing, where novel and unseen classes often emerge during inference. Open-vocabulary image Segmentati…
Mobile-VideoGPT: Fast and Accurate Model for Mobile Video Understanding
Abdelrahman Shaker, Muhammad Maaz, Chenhui Gou +3
Video understanding models often struggle with high computational requirements, extensive parameter counts, and slow inference speed, making them inefficient for practical use. To…
DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning
Fucai Ke, Vijay Kumar B G, Xingjian Leng +6
Visual reasoning (VR), which is crucial in many fields for enabling human-like visual understanding, remains highly challenging. Recently, compositional visual reasoning approaches…
NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning
Zhixi Cai, Fucai Ke, Simindokht Jahangard +4
Visual Grounding (VG) tasks, such as referring expression detection and segmentation tasks are important for linking visual entities to context, especially in complex reasoning tas…
Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering
Meng Wei, Qianyi Wu, Jianmin Zheng +2
Rendering and reconstruction are long-standing topics in computer vision and graphics. Achieving both high rendering quality and accurate geometry is a challenge. Recent advancemen…
TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene
Sandika Biswas, Qianyi Wu, Biplab Banerjee +1
Despite advancements in Neural Implicit models for 3D surface reconstruction, handling dynamic environments with interactions between arbitrary rigid, non-rigid, or deformable enti…