4 papers
Structure-Aligned Protein Language Model
Can Chen, David Heurtel-Depeiges, Robert M. Vernon +3
Protein language models (pLMs) pre-trained on vast protein sequence databases excel at various downstream tasks but often lack the structural knowledge essential for some biologica…
Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data
David Heurtel-Depeiges, Anian Ruoss, Joel Veness +1
Foundation models are strong data compressors, but when accounting for their parameter size, their compression ratios are inferior to standard compression algorithms. Naively reduc…
Listening to the Noise: Blind Denoising with Gibbs Diffusion
David Heurtel-Depeiges, Charles C. Margossian, Ruben Ohana +1
In recent years, denoising problems have become intertwined with the development of deep generative models. In particular, diffusion models are trained like denoisers, and the dist…
Removing Dust from CMB Observations with Diffusion Models
David Heurtel-Depeiges, Blakesley Burkhart, Ruben Ohana +1
In cosmology, the quest for primordial -modes in cosmic microwave background (CMB) observations has highlighted the critical need for a refined model of the Galactic dust foregr…