11 papers · 1 filter
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Praneet Suresh, Jack Stanley, Sonia Joseph +2
Pre-trained transformers have demonstrated remarkable generalization abilities, at times extending beyond the scope of their training data. Yet, real-world deployments often face u…
Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise
Luca Scimeca, Thomas Jiralerspong, Berton Earnshaw +2
Diffusion Probabilistic Models (DPMs) have achieved strong generative performance, yet their inductive biases remain largely implicit. In this work, we aim to build inductive biase…
From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
Praneet Suresh, Jack Stanley, Sonia Joseph +2
As generative AI systems become competent and democratized in science, business, and government, deeper insight into their failure modes now poses an acute need. The occasional vol…
Torsional-GFN: a conditional conformation generator for small molecules
Alexandra Volokhova, Léna Néhale Ezzine, Piotr GaiÅski +5
Generating stable molecular conformations is crucial in several drug discovery applications, such as estimating the binding affinity of a molecule to a target. Recently, generative…
Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models
Siddarth Venkatraman, Mohsin Hasan, Minsu Kim +5
Any well-behaved generative model over a variable can be expressed as a deterministic transformation of an exogenous ('outsourced') Gaussian noise variable $\mathbf{z}…
Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles
Luca Scimeca, Alexander Rubinstein, Damien Teney +2
Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut learning, where a model relies on erroneous…