5 papers · 1 filter
PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Paths
Boyu Chen, Zirui Guo, Zidan Yang +5
Retrieval-augmented generation (RAG) improves the response quality of large language models (LLMs) by retrieving knowledge from external databases. Typical RAG approaches split the…
ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix
Zile Yang, Ling Li, Na Di +5
Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-r…
Large Language Models have Intrinsic Self-Correction Ability
Dancheng Liu, Amir Nassereldine, Ziming Yang +8
Large language models (LLMs) have attracted significant attention for their exceptional abilities in various natural language processing tasks, but they suffer from hallucinations…
Equipping Transformer with Random-Access Reading for Long-Context Understanding
Chenghao Yang, Zi Yang, Nan Hua
Long-context modeling presents a significant challenge for transformer-based large language models (LLMs) due to the quadratic complexity of the self-attention mechanism and issues…
Parameter-Efficient Fine-Tuning With Adapters
Keyu Chen, Yuan Pang, Zi Yang
In the arena of language model fine-tuning, the traditional approaches, such as Domain-Adaptive Pretraining (DAPT) and Task-Adaptive Pretraining (TAPT), although effective, but com…