Publications (25)
Using Large Ensembles of Control Variates for Variational Inference
Tomas Geffner, Justin Domke
Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, sin…
Latent Target Score Matching, with an application to Simulation-Based Inference
Joohwan Ko, Tomas Geffner
Denoising score matching (DSM) for training diffusion models may suffer from high variance at low noise levels. Target Score Matching (TSM) mitigates this when clean data scores ar…
Stochastic Flow Matching for Resolving Small-Scale Physics
Stathi Fotiadis, Noah Brenowitz, Tomas Geffner +4
Conditioning diffusion and flow models have proven effective for super-resolving small-scale details in natural images.However, in physical sciences such as weather, super-resolvin…
Approximation Based Variance Reduction for Reparameterization Gradients
Tomas Geffner, Justin Domke
Flexible variational distributions improve variational inference but are harder to optimize. In this work we present a control variate that is applicable for any reparameterizable…
Compact Policies for Fully-Observable Non-Deterministic Planning as SAT
Tomas Geffner, Hector Geffner
Fully observable non-deterministic (FOND) planning is becoming increasingly important as an approach for computing proper policies in probabilistic planning, extended temporal plan…
Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow
Zhonglin Cao, Mario Geiger, Allan dos Santos Costa +6
Fast and accurate generation of molecular conformers is desired for downstream computational chemistry and drug discovery tasks. Currently, training and sampling state-of-the-art d…
ProtComposer: Compositional Protein Structure Generation with 3D Ellipsoids
Hannes Stark, Bowen Jing, Tomas Geffner +4
We develop ProtComposer to generate protein structures conditioned on spatial protein layouts that are specified via a set of 3D ellipsoids capturing substructure shapes and semant…
Energy-Based Diffusion Language Models for Text Generation
Minkai Xu, Tomas Geffner, Karsten Kreis +5
Despite remarkable progress in autoregressive language models, alternative generative paradigms beyond left-to-right generation are still being actively explored. Discrete diffusio…
Joint control variate for faster black-box variational inference
Xi Wang, Tomas Geffner, Justin Domke
Black-box variational inference performance is sometimes hindered by the use of gradient estimators with high variance. This variance comes from two sources of randomness: Data sub…
Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization
Siyi Gu, Minkai Xu, Alexander Powers +6
Generating ligand molecules for specific protein targets, known as structure-based drug design, is a fundamental problem in therapeutics development and biological discovery. Recen…
Learning Straight Flows by Learning Curved Interpolants
Shiv Shankar, Tomas Geffner
Flow matching models typically use linear interpolants to define the forward/noise addition process. This, together with the independent coupling between noise and target distribut…
La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching
Tomas Geffner, Kieran Didi, Zhonglin Cao +6
Recently, many generative models for de novo protein structure design have emerged. Yet, only few tackle the difficult task of directly generating fully atomistic structures jointl…
Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute
Kieran Didi, Zuobai Zhang, Guoqing Zhou +11
Protein interaction modeling is central to protein design, which has been transformed by machine learning with applications in drug discovery and beyond. In this landscape, structu…
Consistent Synthetic Sequences Unlock Structural Diversity in Fully Atomistic De Novo Protein Design
Danny Reidenbach, Zhonglin Cao, Zuobai Zhang +8
High-quality training datasets are crucial for the development of effective protein design models, but existing synthetic datasets often include unfavorable sequence-structure pair…
Deep End-to-end Causal Inference
Tomas Geffner, Javier Antoran, Adam Foster +9
Causal inference is essential for data-driven decision making across domains such as business engagement, medical treatment and policy making. However, research on causal discovery…
Langevin Diffusion Variational Inference
Tomas Geffner, Justin Domke
Many methods that build powerful variational distributions based on unadjusted Langevin transitions exist. Most of these were developed using a wide range of different approaches a…
Proteina: Scaling Flow-based Protein Structure Generative Models
Tomas Geffner, Kieran Didi, Zuobai Zhang +8
Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale…
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3
Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this…
Compositional Score Modeling for Simulation-based Inference
Tomas Geffner, George Papamakarios, Andriy Mnih
Neural Posterior Estimation methods for simulation-based inference can be ill-suited for dealing with posterior distributions obtained by conditioning on multiple observations, as…
On the Difficulty of Unbiased Alpha Divergence Minimization
Tomas Geffner, Justin Domke
Several approximate inference algorithms have been proposed to minimize an alpha-divergence between an approximating distribution and a target distribution. Many of these algorithm…
Empirical Evaluation of Biased Methods for Alpha Divergence Minimization
Tomas Geffner, Justin Domke
In this paper we empirically evaluate biased methods for alpha-divergence minimization. In particular, we focus on how the bias affects the final solutions found, and how this depe…
Truncated Consistency Models
Sangyun Lee, Yilun Xu, Tomas Geffner +4
Consistency models have recently been introduced to accelerate sampling from diffusion models by directly predicting the solution (i.e., data) of the probability flow ODE (PF ODE)…
A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Tomas Geffner, Justin Domke
Stochastic gradient descent (SGD) is the workhorse of modern machine learning. Sometimes, there are many different potential gradient estimators that can be used. When so, choosing…
MCMC Variational Inference via Uncorrected Hamiltonian Annealing
Tomas Geffner, Justin Domke
Given an unnormalized target distribution we want to obtain approximate samples from it and a tight lower bound on its (log) normalization constant log Z. Annealed Importance Sampl…
Variational Inference with Locally Enhanced Bounds for Hierarchical Models
Tomas Geffner, Justin Domke
Hierarchical models represent a challenging setting for inference algorithms. MCMC methods struggle to scale to large models with many local variables and observations, and variati…