5 papers
EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
Wujiang Xu, Wentian Zhao, Zhenting Wang +6
Training LLM agents in multi-turn environments with sparse rewards, where completing a single task requires 30+ turns of interaction within an episode, presents a fundamental chall…
Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference
Jiayi Yuan, Hao Li, Xinheng Ding +7
Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are…
Prompt-Guided Mask Proposal for Two-Stage Open-Vocabulary Segmentation
Yu-Jhe Li, Xinyang Zhang, Kun Wan +3
We tackle the challenge of open-vocabulary segmentation, where we need to identify objects from a wide range of categories in different environments, using text prompts as our inpu…
Treat Visual Tokens as Text? But Your MLLM Only Needs Fewer Efforts to See
Zeliang Zhang, Phu Pham, Wentian Zhao +6
By treating visual tokens from visual encoders as text tokens, Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse visual understanding tasks,…
Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach
Shijian Deng, Wentian Zhao, Yu-Jhe Li +4
Self-improvement in multimodal large language models (MLLMs) is crucial for enhancing their reliability and robustness. However, current methods often rely heavily on MLLMs themsel…