43 citations · 80 across the 26 of their papers we have counts for
30 papers
CoCo-IR: Contextual Composed Image Retrieval
Shengcao Cao, Tanmaya Shekhar Dabral, Zhongli Ding +6
Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual search…
Think in Latent, Explain in Language: Self-Explainable Latent Reasoning
Dayuan Zhao, Shengcao Cao, Yu-Xiong Wang +1
Latent reasoning has emerged as a powerful alternative to text-based Chain-of-Thought (CoT), offering significant gains in computational efficiency by compressing verbose reasoning…
Vesta: A Generalist Embodied Reasoning Model
Johan Bjorck, Zhiqi Li, Yunze Man +29
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at indiv…
InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions
Sirui Xu, Samuel Schulter, Morteza Ziyadi +4
Humans rarely plan whole-body interactions with objects at the level of explicit whole-body movements. High-level intentions, such as affordance, define the goal, while coordinated…
PPTArena: A Benchmark for PowerPoint Editing
Michael Ofengenden, Yunze Man, Ziqi Pang +2
We introduce PPTArena, a benchmark for PowerPoint editing that evaluates how agents modify real slides from natural-language instructions. Unlike benchmarks that rely on image-PDF…
LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
Yunze Man, Shihao Wang, Guowen Zhang +7
To act in the world, a model must name what it sees and know where it is in 3D. Today's vision-language models (VLMs) excel at open-ended 2D description and grounding, yet multi-ob…
BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learning
Qiusi Zhan, Hyeonjeong Ha, Rui Yang +7
Recent advances in Vision-Language Models (VLMs) have propelled embodied agents by enabling direct perception, reasoning, and planning task-oriented actions from visual inputs. How…
MoReact: Generating Reactive Motion from Textual Descriptions
Xiyan Xu, Sirui Xu, Yu-Xiong Wang +1
Modeling and generating human reactions poses a significant challenge with broad applications for computer vision and human-computer interaction. Existing methods either treat mult…
Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration
Sirui Xu, Yu-Wei Chao, Liuyu Bian +4
Hand-object motion-capture (MoCap) repositories offer large-scale, contact-rich demonstrations and hold promise for scaling dexterous robotic manipulation. Yet demonstration inaccu…
InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation
Sirui Xu, Dongting Li, Yucheng Zhang +9
While large-scale human motion capture datasets have advanced human motion generation, modeling and generating dynamic 3D human-object interactions (HOIs) remain challenging due to…
Refer to Any Segmentation Mask Group With Vision-Language Prompts
Shengcao Cao, Zijun Wei, Jason Kuen +6
Recent image segmentation models have advanced to segment images into high-quality masks for visual entities, and yet they cannot provide comprehensive semantic understanding for c…
Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought
Yunze Man, De-An Huang, Guilin Liu +6
Recent advances in multimodal large language models (MLLMs) have demonstrated remarkable capabilities in vision-language tasks, yet they often struggle with vision-centric scenario…
InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions
Sirui Xu, Hung Yu Ling, Yu-Xiong Wang +1
Achieving realistic simulations of humans interacting with a wide range of objects has long been a fundamental goal. Extending physics-based motion imitation to complex human-objec…
Emergent Visual Grounding in Large Multimodal Models Without Grounding Supervision
Shengcao Cao, Liang-Yan Gui, Yu-Xiong Wang
Current large multimodal models (LMMs) face challenges in grounding, which requires the model to relate language components to visual entities. Contrary to the common practice that…
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding
Yunze Man, Shuhong Zheng, Zhipeng Bao +3
Complex 3D scene understanding has gained increasing attention, with scene encoding strategies playing a crucial role in this success. However, the optimal scene encoding strategie…
Floating No More: Object-Ground Reconstruction from a Single Image
Yunze Man, Yichen Sheng, Jianming Zhang +2
Recent advancements in 3D object reconstruction from single images have primarily focused on improving the accuracy of object shapes. Yet, these techniques often fail to accurately…
Situational Awareness Matters in 3D Vision Language Reasoning
Yunze Man, Liang-Yan Gui, Yu-Xiong Wang
Being able to carry out complicated vision language reasoning tasks in 3D space represents a significant milestone in developing household robots and human-centered embodied AI. In…
SOHES: Self-supervised Open-world Hierarchical Entity Segmentation
Shengcao Cao, Jiuxiang Gu, Jason Kuen +7
Open-world entity segmentation, as an emerging computer vision task, aims at segmenting entities in images without being restricted by pre-defined classes, offering impressive gene…
InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction
Sirui Xu, Ziyin Wang, Yu-Xiong Wang +1
Text-conditioned human motion generation has experienced significant advancements with diffusion models trained on extensive motion capture data and corresponding textual annotatio…
HASSOD: Hierarchical Adaptive Self-Supervised Object Detection
Shengcao Cao, Dhiraj Joshi, Liang-Yan Gui +1
The human visual perception system demonstrates exceptional capabilities in learning without explicit supervision and understanding the part-to-whole composition of objects. Drawin…
Aligning Large Multimodal Models with Factually Augmented RLHF
Zhiqing Sun, Sheng Shen, Shengcao Cao +9
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounde…
InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion
Sirui Xu, Zhengyuan Li, Yu-Xiong Wang +1
This paper addresses a novel task of anticipating 3D human-object interactions (HOIs). Most existing research on HOI synthesis lacks comprehensive whole-body interactions with dyna…
Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
Shengcao Cao, Mengtian Li, James Hays +3
Resource-constrained perception systems such as edge computing and vision-for-robotics require vision models to be both accurate and lightweight in computation and memory usage. Wh…
Stochastic Multi-Person 3D Motion Forecasting
Sirui Xu, Yu-Xiong Wang, Liang-Yan Gui
This paper aims to deal with the ignored real-world complexities in prior work on human motion forecasting, emphasizing the social properties of multi-person motion, the diversity…