9 papers · 1 filter
Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
Jiachen Liang, Ruibing Hou, Minyang Hu +3
Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their effici…
KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledge
Zaifei Yang, Hong Chang, Ruibing Hou +2
The molecular large language models have garnered widespread attention due to their promising potential on molecular applications. However, current molecular large language models…
Revisiting Multimodal Positional Encoding in Vision-Language Models
Jie Huang, Xuejing Liu, Sibo Song +4
Multimodal position encoding is essential for vision-language models, yet there has been little systematic investigation into multimodal position encoding. We conduct a comprehensi…
MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing
Ruibing Hou, Mingshuang Luo, Hongyu Pan +2
This paper proposes MotionVerse, a unified framework that harnesses the capabilities of Large Language Models (LLMs) to comprehend, generate, and edit human motion in both single-p…
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
Keliang Li, Hongze Shen, Hao Shi +9
The aspiration for artificial general intelligence, fueled by the rapid progress of multimodal models, demands human-comparable performance across diverse environments. We propose…
unCLIP: Improving CLIP's Visual Detail Capturing Ability via Inverting unCLIP
Yinqi Li, Jiahe Zhao, Hong Chang +3
Contrastive Language-Image Pre-training (CLIP) has become a foundation model and has been applied to various vision and multimodal tasks. However, recent works indicate that CLIP f…
DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks
Yinqi Li, Hong Chang, Ruibing Hou +2
Diffusion models have shown remarkable progress in various generative tasks such as image and video generation. This paper studies the problem of leveraging pretrained diffusion mo…
HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding
Jiahe Zhao, Ruibing Hou, Zejie Tian +2
We propose a new task to benchmark human-in-scene understanding for embodied agents: Human-In-Scene Question Answering (HIS-QA). Given a human motion within a 3D scene, HIS-QA requ…
MATS: An Audio Language Model under Text-only Supervision
Wen Wang, Ruibing Hou, Hong Chang +2
Large audio-language models (LALMs), built upon powerful Large Language Models (LLMs), have exhibited remarkable audio comprehension and reasoning capabilities. However, the traini…