2 citations · 4 across the 6 of their papers we have counts for
6 papers
Reclaiming Low-Quality Instruction Data with Rule-Guided LLM Reconstruction
Zile Yang, Ling Li, Na Di +5
Data selection for supervised fine-tuning (SFT) often discards low-scoring instruction examples that may retain useful supervision. We propose a rule-guided LLM reconstruction fram…
PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Paths
Boyu Chen, Zirui Guo, Zidan Yang +5
Retrieval-augmented generation (RAG) improves the response quality of large language models (LLMs) by retrieving knowledge from external databases. Typical RAG approaches split the…
Large Language Models have Intrinsic Self-Correction Ability
Dancheng Liu, Amir Nassereldine, Ziming Yang +8
Large language models (LLMs) have attracted significant attention for their exceptional abilities in various natural language processing tasks, but they suffer from hallucinations…
Equipping Transformer with Random-Access Reading for Long-Context Understanding
Chenghao Yang, Zi Yang, Nan Hua
Long-context modeling presents a significant challenge for transformer-based large language models (LLMs) due to the quadratic complexity of the self-attention mechanism and issues…
Parameter-Efficient Fine-Tuning With Adapters
Keyu Chen, Yuan Pang, Zi Yang
In the arena of language model fine-tuning, the traditional approaches, such as Domain-Adaptive Pretraining (DAPT) and Task-Adaptive Pretraining (TAPT), although effective, but com…
Attendre: Wait To Attend By Retrieval With Evicted Queries in Memory-Based Transformers for Long Context Processing
Zi Yang, Nan Hua
As LLMs have become capable of processing more complex types of inputs, researchers have recently studied how to efficiently and affordably process possibly arbitrarily long sequen…