2 papers
cs.CL2026
Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models
Ayan Sengupta, Vaibhav Seth, Tanmoy Chakraborty
Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implicitly assuming the teacher to…
cs.LG2024
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
Ayan Sengupta, Vaibhav Seth, Arinjay Pathak +5
Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approac…