6 papers · 1 filter
Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment
Yucong Huang, Xiucheng Li, Kaiqi Zhao +1
Standard RLHF relies on transitive scalar rewards, failing to capture the cyclic nature of human preferences. While some approaches like the General Preference Model (GPM) address…
Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoning
Zhuoyuan Hao, Zhuo Li, Wu Li +3
Test-time compute allocation in large reasoning models (LRMs) is widely used and has applications in mathematical problem solving, code synthesis, and planning. Recent work has add…
Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries
Yin Wu, Quanyu Long, Jing Li +2
Retrieval-augmented generation (RAG) is a paradigm that augments large language models (LLMs) with external knowledge to tackle knowledge-intensive question answering. While severa…
Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing
Yifan Lu, Jing Li, Yigeng Zhou +7
Large language models (LLMs) exhibit impressive language capabilities but remain vulnerable to malicious prompts and jailbreaking attacks. Existing knowledge editing methods for LL…
Multi-objective Large Language Model Alignment with Hierarchical Experts
Zhuo Li, Guodong Du, Weiyang Guo +8
Aligning large language models (LLMs) to simultaneously satisfy multiple objectives remains a significant challenge, especially given the diverse and often conflicting nature of hu…
Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decoupling
Junlin Li, Guodong DU, Jing Li +8
Fine-tuning Large Language Models (LLMs) with multimodal encoders on modality-specific data expands the modalities that LLMs can handle, leading to the formation of Multimodal LLMs…