15 papers
CarryOn: A Dual-Loop Harness for Personalized and Reliable Robot Assistance
Zixuan Wang, Yuqi Liu, Yuxin Chen +7
While generalist vision-language-action (VLA) models have enabled diverse open-world manipulation, real-world robotic assistance requires two critical capabilities they currently l…
Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Senqiao Yang, Chengyao Wang, Yuxin Chen +13
Scaling robot data is crucial for building generalist Vision-Language-Action (VLA) models, yet robot trajectories are harder to scale than web-scale image-text data because embodie…
MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
Yuqi Liu, Shengju Qian, Tianyuan Qu +5
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with l…
FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
Zehua Pei, Hui-Ling Zhen, Xianzhi Yu +3
Large language models can now process increasingly long inputs, yet their ability to effectively use information spread across long contexts remains limited. We trace this gap to h…
RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing
Tianyuan Qu, Lei Ke, Xiaohang Zhan +6
Instruction-based image editing enables natural-language control over visual modifications, yet existing models falter under Instruction-Visual Complexity (IV-Complexity), where in…
Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents
Mingkang Zhu, Xi Chen, Bei Yu +2
Large language model (LLM) agents increasingly rely on external tools such as search engines to solve complex, multi-step problems, and reinforcement learning (RL) has become a key…
From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
Mingkang Zhu, Xi Chen, Bei Yu +2
Large reasoning models (LRMs) generate intermediate reasoning traces before producing final answers, yielding strong gains on multi-step and mathematical tasks. Yet aligning LRMs w…
ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models
Yuqi Liu, Liangyu Chen, Jiazhen Liu +4
Post-training Large Vision-and-Language Models (LVLMs) typically involves Supervised Fine-Tuning (SFT) for knowledge injection or Reinforcement Learning with Verifiable Rewards (RL…
MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech
Chengyao Wang, Zhisheng Zhong, Bohao Peng +7
We present MGM-Omni, a unified Omni LLM for omni-modal understanding and expressive, long-horizon speech generation. Unlike cascaded pipelines that isolate speech synthesis, MGM-Om…
TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
Mingkang Zhu, Xi Chen, Zhongdao Wang +3
Recent advancements in reinforcement learning from human feedback have shown that utilizing fine-grained token-level reward models can substantially enhance the performance of Prox…
RTime-QA: A Benchmark for Atomic Temporal Event Understanding in Large Multi-modal Models
Yuqi Liu, Qin Jin, Tianyuan Qu +4
Understanding accurate atomic temporal event is essential for video comprehension. However, current video-language benchmarks often fall short to evaluate Large Multi-modal Models'…
VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning
Yuqi Liu, Tianyuan Qu, Zhisheng Zhong +4
Large vision-language models exhibit inherent capabilities to handle diverse visual perception tasks. In this paper, we introduce VisionReasoner, a unified framework capable of rea…
Modular Customization of Diffusion Models via Blockwise-Parameterized Low-Rank Adaptation
Mingkang Zhu, Xi Chen, Zhongdao Wang +3
Recent diffusion model customization has shown impressive results in incorporating subject or style concepts with a handful of images. However, the modular composition of multiple…
Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
Yuqi Liu, Bohao Peng, Zhisheng Zhong +4
Traditional methods for reasoning segmentation rely on supervised fine-tuning with categorical labels and simple descriptions, limiting its out-of-domain generalization and lacking…
Enhancing LLM Knowledge Learning through Generalization
Mingkang Zhu, Xi Chen, Zhongdao Wang +3
As Large language models (LLMs) are increasingly deployed in diverse applications, faithfully integrating evolving factual knowledge into these models remains a critical challenge.…