8 papers
MIPS: Mitigating Mode Collapse in Reinforcement Learning for Large Outcome Spaces
Armaan Sandhu, Abhijeet Sinha, Dianbo Liu
Expected-reward maximization in reinforcement learning concentrates probability mass on a few outcomes, even in tasks that require many distinct, high-quality solutions, a failure…
Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training
Peng Sun, Yi Yang, Antong Zhang +7
Data selection in supervised fine-tuning aims to select a small set of effective samples from large-scale candidate data, reducing training cost while preserving model performance.…
Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
Abhijeet Sinha, Sundari Elango, Dianbo Liu
Many reinforcement learning (RL) problems admit multiple terminal solutions of comparable quality, where the goal is not to identify a single optimum but to represent a diverse set…
Deconstructing Generative Diversity: An Information Bottleneck Analysis of Discrete Latent Generative Models
Yudi Wu, Wenhao Zhao, Dianbo Liu
Generative diversity varies significantly across discrete latent generative models such as AR, MIM, and Diffusion. We propose a diagnostic framework, grounded in Information Bottle…
CodeUnlearn: Amortized Zero-Shot Machine Unlearning in Language Models Using Discrete Concept
YuXuan Wu, Bonaventure F. P. Dossou, Dianbo Liu
Large Language Models (LLMs) offer extensive knowledge across various domains, but they may inadvertently memorize sensitive, unauthorized, or malicious data, such as personal info…
Brain-inspired continual pre-trained learner via silent synaptic consolidation
Xuming Ran, Juntao Yao, Yusong Wang +2
Pre-trained models have demonstrated impressive generalization capabilities, yet they remain vulnerable to catastrophic forgetting when incrementally trained on new tasks. Existing…
BarlowTwins-CXR : Enhancing Chest X-Ray abnormality localization in heterogeneous data with cross-domain self-supervised learning
Haoyue Sheng, Linrui Ma, Jean-Francois Samson +1
Background: Chest X-ray imaging-based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and t…
Unsupervised Concept Discovery Mitigates Spurious Correlations
Md Rifat Arefin, Yan Zhang, Aristide Baratin +4
Models prone to spurious correlations in training data often produce brittle predictions and introduce unintended biases. Addressing this challenge typically involves methods relyi…