3 citations · 6 across the 27 of their papers we have counts for
35 papers
JANUS: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models
Mengdi Liu, Chenghao Jia, Hong Chang +3
Long-horizon autonomous research has shown promise in accelerating scientific progress through sustained cycles of hypothesis generation, implementation, and validation. However, t…
Probe to Act: Elevating Browser-Use Agent via Active Visual Probing
Keliang Li, Heng Wang, Chen Hu +3
Browser-use agents require seamless alignment between structured web metadata and visual information, while preserving relevant context across long interactions. Existing interface…
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
Zejie Tian, Ruibing Hou, Bingpeng Ma +2
Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-…
EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting
Wen Wang, Ruibing Hou, Hong Chang +2
Forecasting dexterous hand motions from egocentric observations is fundamental to intelligent interactive systems. Existing VLM-based methods typically map observations directly to…
EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
Xuanyu Ge, Zhongqi Wang, Jie Zhang +2
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks. Existing defense methods predom…
Component-Based Out-of-Distribution Detection
Wenrui Liu, Hong Chang, Ruibing Hou +2
Out-of-Distribution (OOD) detection requires sensitivity to subtle shifts without overreacting to natural In-Distribution (ID) diversity. However, from the viewpoint of detection g…
EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generation
Ruibing Hou, Mingyue Zhou, Yuwei Gui +5
Faithfully modeling human behavior in dynamic environments is a foundational challenge for embodied intelligence. While conditional motion synthesis has achieved significant advanc…
Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
Jiachen Liang, Ruibing Hou, Minyang Hu +3
Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their effici…
KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledge
Zaifei Yang, Hong Chang, Ruibing Hou +2
The molecular large language models have garnered widespread attention due to their promising potential on molecular applications. However, current molecular large language models…
MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing
Ruibing Hou, Mingshuang Luo, Hongyu Pan +2
This paper proposes MotionVerse, a unified framework that harnesses the capabilities of Large Language Models (LLMs) to comprehend, generate, and edit human motion in both single-p…
GLip: A Global-Local Integrated Progressive Framework for Robust Visual Speech Recognition
Tianyue Wang, Shuang Yang, Shiguang Shan +1
Visual speech recognition (VSR), also known as lip reading, is the task of recognizing speech from silent video. Despite significant advancements in VSR over recent decades, most e…
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
Keliang Li, Hongze Shen, Hao Shi +9
The aspiration for artificial general intelligence, fueled by the rapid progress of multimodal models, demands human-comparable performance across diverse environments. We propose…
ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search
Mengdi Liu, Xiaoxue Cheng, Zhangyang Gao +4
Designing protein sequences that fold into a target 3D structure, known as protein inverse folding, is a fundamental challenge in protein engineering. While recent deep learning me…
unCLIP: Improving CLIP's Visual Detail Capturing Ability via Inverting unCLIP
Yinqi Li, Jiahe Zhao, Hong Chang +3
Contrastive Language-Image Pre-training (CLIP) has become a foundation model and has been applied to various vision and multimodal tasks. However, recent works indicate that CLIP f…
Jodi: Unification of Visual Generation and Understanding via Joint Modeling
Yifeng Xu, Zhenliang He, Meina Kan +2
Visual generation and understanding are two deeply interconnected aspects of human intelligence, yet they have been traditionally treated as separate tasks in machine learning. In…
Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling
Xiaolong Tang, Meina Kan, Shiguang Shan +1
Safe and feasible trajectory planning is critical for real-world autonomous driving systems. However, existing learning-based planners rely heavily on expert demonstrations, which…
DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks
Yinqi Li, Hong Chang, Ruibing Hou +2
Diffusion models have shown remarkable progress in various generative tasks such as image and video generation. This paper studies the problem of leveraging pretrained diffusion mo…
G2PDiffusion: Cross-Species Genotype-to-Phenotype Prediction via Evolutionary Diffusion
Mengdi Liu, Zhangyang Gao, Hong Chang +3
Understanding how genes influence phenotype across species is a fundamental challenge in genetic engineering, which will facilitate advances in various fields such as crop breeding…
HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding
Jiahe Zhao, Ruibing Hou, Zejie Tian +2
We propose a new task to benchmark human-in-scene understanding for embodied agents: Human-In-Scene Question Answering (HIS-QA). Given a human motion within a 3D scene, HIS-QA requ…
MATS: An Audio Language Model under Text-only Supervision
Wen Wang, Ruibing Hou, Hong Chang +2
Large audio-language models (LALMs), built upon powerful Large Language Models (LLMs), have exhibited remarkable audio comprehension and reasoning capabilities. However, the traini…
RefHCM: A Unified Model for Referring Perceptions in Human-Centric Scenarios
Jie Huang, Ruibing Hou, Jiahe Zhao +2
Human-centric perceptions play a crucial role in real-world applications. While recent human-centric works have achieved impressive progress, these efforts are often constrained to…
UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models
Jiachen Liang, Ruibing Hou, Minyang Hu +3
Pre-trained vision-language models (e.g., CLIP) have shown powerful zero-shot transfer capabilities. But they still struggle with domain shifts and typically require labeled data t…
UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing
Yiheng Li, Ruibing Hou, Hong Chang +2
Human pose plays a crucial role in the digital age. While recent works have achieved impressive progress in understanding and generating human poses, they often support only a sing…
HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding
Keliang Li, Zaifei Yang, Jiahe Zhao +5
The significant advancements in visual understanding and instruction following from Multimodal Large Language Models (MLLMs) have opened up more possibilities for broader applicati…