7 papers
ARO: A New Lens On Matrix Optimization For Large Models
Wenbo Gong, Javier Zazo, Qijun Luo +3
Matrix-based optimizers have attracted growing interest for improving LLM training efficiency, with significant progress centered on orthogonalization/whitening based methods. Whil…
Low Rank Comes with Low Security: Gradient Assembly Poisoning Attacks against Distributed LoRA-based LLM Systems
Yueyan Dong, Minghui Xu, Qin Hu +5
Low-Rank Adaptation (LoRA) has become a popular solution for fine-tuning large language models (LLMs) in federated settings, dramatically reducing update costs by introducing train…
Multi-hop Reasoning via Early Knowledge Alignment
Yuxin Wang, Shicheng Fang, Bo Wang +4
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for Large Language Models (LLMs) to address knowledge-intensive queries requiring domain-specific or up-to-d…
Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledge
Qi Luo, Xiaonan Li, Junqi Dai +2
Retrieval-Augmented Generation has shown remarkable results to address Large Language Models' hallucinations, which usually uses a large external corpus to supplement knowledge to…
Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning
Qi Luo, Xiaonan Li, Tingshuo Fan +2
Retrieval-augmented generation (RAG) has emerged as a leading approach to reducing hallucinations in large language models (LLMs). Current RAG evaluation benchmarks primarily focus…
MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
Qi Luo, Xiaonan Li, Yuxin Wang +4
Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data, resulting in factual inaccuracies and weak adaptability to new…