14 papers
Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos
Wendong Bu, Tinghong Ye, Qizhou Wang +21
Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal i…
Deep Search in Real-World Web Environments
Xiaomin Yu, Hongliang Chen, Zhe Huang +22
Despite substantial progress in Deep Search agents, their ability to conduct end-to-end research in real-world open Web environments remains insufficiently evaluated. Unlike static…
Dyad: Extending Large Language Models with Native Typed Decision-Making
Yundaichuan Zhan, Weishi Wang, Wenbiao Liu +5
We study how to build more capable general-purpose agents by extending large language models (LLMs) with native typed decision-making. We introduce Dyad, an architecture that augme…
How Far Do Persona Effects Generalize in Language Models?
Yufan Zhou, Yuxuan Liu, Enze Ma +6
Persona prompts ask language models to answer as particular kinds of people. We test whether relationships learned from these effects predict responses to new questions and remain…
Visual Attention Faithfulness in Vision-Language Models is Heterogeneous
Xurui Song, Weishi Wang, Zhongqi Yue +5
Whether attention weights faithfully reflect model reasoning has been actively debated in NLP, yet this question remains largely unexplored for the visual modality in Vision-Langua…
SCOPE: Evolving Symbolic World for Planning in Open-Ended Environments
Yundaichuan Zhan, Minghe Gao, Zhongqi Yue +7
Recent works have explored integrating Vision-Language Models (VLMs) with classical planners that rely on symbolic representations of planning problems to generate long-horizon pla…
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
Jianbo Lin, Xiaomin Yu, Yi Xin +7
Large language model-based agents make mistakes, yet critique can often guide the same model toward correct behavior. However, when critique is removed, the model may fail again on…
When are radiology reports useful for training medical image classifiers?
Herman Bergström, Zhongqi Yue, Fredrik D. Johansson
Medical images used to train machine learning models are often accompanied by radiology reports containing rich expert annotations. However, relying on these reports as inputs for…
Expanding the Action Space of LLMs to Reason Beyond Language
Zhongqi Yue, Weishi Wang, Yundaichuan Zhan +3
Large Language Models (LLMs) are powerful reasoners in natural language, but their actions are typically confined to outputting vocabulary tokens. As a result, interactions with ex…
Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning
Bohan Wang, Zhongqi Yue, Fengda Zhang +15
We completely discard the conventional spatial prior in image representation and introduce a novel discrete visual tokenizer: Self-consistency Tokenizer (Selftok). At its design co…
Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learning
Wang Lin, Liyu Jia, Wentao Hu +6
Despite recent progress in video generation, producing videos that adhere to physical laws remains a significant challenge. Traditional diffusion-based methods struggle to extrapol…
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens
Kaihang Pan, Wang Lin, Zhongqi Yue +6
Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation by combining LLM and diffusion models, the state-of-the-art in each ta…
Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program
Minghe Gao, Xuqi Liu, Zhongqi Yue +7
Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal…
Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativeness
Qifan Yu, Zhebei Shen, Zhongqi Yue +7
Instruction tuning fine-tunes pre-trained Multi-modal Large Language Models (MLLMs) to handle real-world tasks. However, the rapid expansion of visual instruction datasets introduc…