5 papers
Stellar: Scalable Multimodal Document Retrieval for Natural Language Queries
Yuxiang Guo, Zhonghao Hu, Yuren Mao +5
Multimodal document retrieval--selecting the most relevant multimodal document from a large corpus to answer a natural language query--plays an essential role in Retrieval-Augmente…
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
Yu Huang, Zihua Zhao, Zhaoxin Huan +9
The open-ended generation in LLMs usually requires multi-dimensional rubrics to adequately assess quality and guide the improvement of reinforcement learning. However, a critical d…
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
Zebin You, Shen Nie, Xiaolu Zhang +5
In this work, we introduce LLaDA-V, a purely diffusion-based Multimodal Large Language Model (MLLM) that integrates visual instruction tuning with masked diffusion models, represen…
One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems
Zuoli Tang, Zhaoxin Huan, Zihao Li +6
Sequential recommendation systems aim to predict users' next likely interaction based on their history. However, these systems face data sparsity and cold-start problems. Utilizing…
An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
Youshao Xiao, Zhenglei Zhou, Fagui Mao +6
Recently, ChatGPT or InstructGPT like large language models (LLM) has made a significant impact in the AI world. Many works have attempted to reproduce the complex InstructGPT's tr…