6 papers · 1 filter
Distilling Safe LLM Systems via Soft Prompts for On Device Settings
Motasem Alfarra, Cristina Pinneri, Dana Kianfar +2
Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide ef…
Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
Usman Anwar, Tim Bakker, Dana Kianfar +2
Chain-of-thought (CoT) monitors are LLM-based systems that analyze reasoning traces to detect when outputs may exhibit attributes of interest, such as test-hacking behavior during…
Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models
Julianna Piskorz, Cristina Pinneri, Alvaro Correia +3
Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in princ…
Fundamental bounds on efficiency-confidence trade-off for transductive conformal prediction
Arash Behboodi, Alvaro H. C. Correia, Fabio Valerio Massoli +1
Transductive conformal prediction addresses the simultaneous prediction for multiple data points. Given a desired confidence level, the objective is to construct a prediction set t…
Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence
Dharmesh Tailor, Alvaro H. C. Correia, Eric Nalisnick +1
Uncertainty quantification is an important prerequisite for the deployment of deep learning models in safety-critical areas. Yet, this hinges on the uncertainty estimates being use…
Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Data
Alvaro H. C. Correia, Christos Louizos
Conformal prediction is a distribution-free uncertainty quantification method that has gained popularity in the machine learning community due to its finite-sample guarantees and e…