148 citations · 316 across the 12 of their papers we have counts for
12 papers
Dense Structural Compression of Transformers via Gauge-Correct Channel Removal
Jed A. Duersch, Naïm Es-Sebbani, Nathanaël Haas +1
Inference energy per token drives the cost and carbon footprint of deployed transformers. It is dominated by dense matrix products that incur fused multiply-accumulate (FMA) operat…
Fourier Transformers for Latent Crystallographic Diffusion and Generative Modeling
Jed A. Duersch, Elohan Veillon, Astrid Klipfel +2
The discovery of new crystalline materials calls for generative models that handle periodic boundary conditions, crystallographic symmetries, and physical constraints, while scalin…
Curvature in the Looking-Glass: Optimal Methods to Exploit Curvature of Expectation in the Loss Landscape
Jed A. Duersch, Tommie A. Catanach, Alexander Safonov +1
Harnessing the local topography of the loss landscape is a central challenge in advanced optimization tasks. By accounting for the effect of potential parameter changes, we can alt…
Projective Integral Updates for High-Dimensional Variational Inference
Jed A. Duersch
Variational inference is an approximation framework for Bayesian inference that seeks to improve quantified uncertainty in predictions by optimizing a simplified distribution over…
Variational Kalman Filtering with Hinf-Based Correction for Robust Bayesian Learning in High Dimensions
Niladri Das, Jed A. Duersch, Thomas A. Catanach
In this paper, we address the problem of convergence of sequential variational inference filter (VIF) through the application of a robust variational objective and Hinf-norm based…
Adaptive n-ary Activation Functions for Probabilistic Boolean Logic
Jed A. Duersch, Thomas A. Catanach, Niladri Das
Balancing model complexity against the information contained in observed data is the central challenge to learning. In order for complexity-efficient models to exist and be discove…
Parsimonious Inference
Jed A. Duersch, Thomas A. Catanach
Bayesian inference provides a uniquely rigorous approach to obtain principled justification for uncertainty in predictions, yet it is difficult to articulate suitably general prior…
Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations
Jed A. Duersch, Ming Gu
Rank-revealing matrix decompositions provide an essential tool in spectral analysis of matrices, including the Singular Value Decomposition (SVD) and related low-rank approximation…
Generalizing Information to the Evolution of Rational Belief
Jed A. Duersch, Thomas A. Catanach
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an ou…
Generalized Canonical Polyadic Tensor Decomposition
David Hong, Tamara G. Kolda, Jed A. Duersch
Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work devel…
A robust and efficient implementation of LOBPCG
Jed A. Duersch, Meiyue Shao, Chao Yang +1
Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is widely used to compute eigenvalues of large sparse symmetric matrices. The algorithm can suffer from numerical i…
Randomized QR with Column Pivoting
Jed A. Duersch, Ming Gu
The dominant contribution to communication complexity in factorizing a matrix using QR with column pivoting is due to column-norm updates that are required to process pivot decisio…