5 papers
MUSE: A Unified Agentic Harness for MLLMs
Jianglin Lu, Hailing Wang, Xu Ma +4
Despite rapid progress, multimodal large language models (MLLMs) still fail on tasks that humans solve effortlessly, such as navigating a grid maze from a screenshot or selecting t…
Boosting Large Language Models with Mask Fine-Tuning
Mingyuan Zhang, Yue Bai, Huan Wang +4
The large language model (LLM) is typically integrated into the mainstream optimization protocol. No work has questioned whether maintaining the model integrity is \textit{indispen…
Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models
Mingyuan Zhang, Yue Bai, Yifan Wang +2
Explorations in fine-tuning Vision-Language Models (VLMs), such as Low-Rank Adaptation (LoRA) from Parameter Efficient Fine-Tuning (PEFT), have made impressive progress. However, m…
Trajectory Prediction Meets Large Language Models: A Survey
Yi Xu, Ruining Yang, Yitian Zhang +5
Recent advances in large language models (LLMs) have sparked growing interest in integrating language-driven techniques into trajectory prediction. By leveraging their semantic and…
Cautious Next Token Prediction
Yizhou Wang, Lingzhi Zhang, Yue Bai +7
Next token prediction paradigm has been prevailing for autoregressive models in the era of LLMs. The current default sampling choice for popular LLMs is temperature scaling togethe…