11 papers
A Survey on LLM-based Conversational User Simulation
Bo Ni, Leyao Wang, Yu Wang +27
User simulation has long played a vital role in computer science due to its potential to support a wide range of applications. Language, as the primary medium of human communicatio…
VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
Zhehao Zhang, Ryan Rossi, Tong Yu +7
While vision-language models (VLMs) have demonstrated remarkable performance across various tasks combining textual and visual information, they continue to struggle with fine-grai…
CoMMIT: Coordinated Multimodal Instruction Tuning
Xintong Li, Junda Wu, Tong Yu +6
Instruction tuning in multimodal large language models (MLLMs) generally involves cooperative learning between a backbone LLM and a feature encoder of non-text input modalities. Th…
Personalization of Large Language Models: A Survey
Zhehao Zhang, Ryan A. Rossi, Branislav Kveton +18
Personalization of Large Language Models (LLMs) has recently become increasingly important with a wide range of applications. Despite the importance and recent progress, most exist…
CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
Leitian Tao, Xiang Chen, Tong Yu +4
Large Language Models (LLMs) have revolutionized code generation but require significant resources and often over-generalize, limiting their task-specific efficiency. Fine-tuning s…
A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations
Li Li, Peilin Cai, Ryan A. Rossi +21
We present PersonaConvBench, a large-scale benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs). Unlike exis…