3 papers
cs.CL2025
Enabling Approximate Joint Sampling in Diffusion LMs
Parikshit Bansal, Sujay Sanghavi
In autoregressive language models, each token is sampled by conditioning on all the past tokens; the overall string has thus been sampled from the correct underlying joint distribu…
cs.LG2025
Context-Free Synthetic Data Mitigates Forgetting
Parikshit Bansal, Sujay Sanghavi
Fine-tuning a language model often results in a degradation of its existing performance on other tasks, due to a shift in the model parameters; this phenomenon is often referred to…
cs.LG2024
Understanding Self-Supervised Learning via Gaussian Mixture Models
Parikshit Bansal, Ali Kavis, Sujay Sanghavi
Self-supervised learning attempts to learn representations from un-labeled data; it does so via a loss function that encourages the embedding of a point to be close to that of its…