7 citations · 11 across the 13 of their papers we have counts for
6 papers · 1 filter
Sculpting the Latent Space: Disentanglement with WAE-MMD and Programmable Priors
Quentin Fruytier, Akshay Malhotra, Shahab Hamidi-Rad +3
Learning disentangled representations, where semantic features are independently captured in a latent space, is dominated by Variational Auto Encoders (VAE) and Adversarial Trainin…
HiSpec: Hierarchical Speculative Decoding for LLMs
Avinash Kumar, Sujay Sanghavi, Poulami Das
Speculative decoding accelerates LLM inference by using a smaller draft model to speculate tokens that a larger target model verifies. Verification is often the bottleneck (e.g. ve…
Enabling Approximate Joint Sampling in Diffusion LMs
Parikshit Bansal, Sujay Sanghavi
In autoregressive language models, each token is sampled by conditioning on all the past tokens; the overall string has thus been sampled from the correct underlying joint distribu…
SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
Jiacheng Lin, Zhongruo Wang, Kun Qian +14
Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach to adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their gene…
Context-Free Synthetic Data Mitigates Forgetting
Parikshit Bansal, Sujay Sanghavi
Fine-tuning a language model often results in a degradation of its existing performance on other tasks, due to a shift in the model parameters; this phenomenon is often referred to…
Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
Sunny Sanyal, Hayden Prairie, Rudrajit Das +2
Fine-tuning a pre-trained model on a downstream task often degrades its original capabilities, a phenomenon known as "catastrophic forgetting". This is especially an issue when one…