1 citations · 1 across the 5 of their papers we have counts for
3 papers · 1 filter
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models
Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi +7
RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual…
KItCAT: Knowledge Injection via Input Corruption for Auto-regressive Training
Meghanadh Pulivarthi, Kushagra Bhushan, Vineet Kumar +5
LLMs acquire vast amounts of knowledge during pre-training, but often lack the specialized knowledge needed to answer questions from niche sources such as manuals or technical docu…
How Much is Too Much? Exploring LoRA Rank Trade-offs for Retaining Knowledge and Domain Robustness
Darshita Rathore, Vineet Kumar, Chetna Bansal +1
Large language models are increasingly adapted to downstream tasks through fine-tuning. Full supervised fine-tuning (SFT) and parameter-efficient fine-tuning (PEFT) methods, such a…