312 citations · 1.2k across the 54 of their papers we have counts for
60 papers
The Tilted Sampling Problem: Optimism, Posterior Sampling, and Logarithmic Regret
Mohammad Pedramfar, Siamak Ravanbakhsh
Many modern learning systems are used not to select a single best action, but to sample diverse high-quality outputs. Examples include entropy-regularized reinforcement learning, R…
The Role of Symmetry in Optimizing Overparameterized Networks
Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2
Overparameterization is central to the success of deep learning, yet the mechanisms by which it improves optimization remain incompletely understood. We analyze weight-space symmet…
The Expressive Limits of Diagonal SSMs for State-Tracking
Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh +1
State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-p…
Inverting Data Transformations via Diffusion Sampling
Jinwoo Kim, Sékou-Oumar Kaba, Jiyun Park +2
We study the problem of transformation inversion on general Lie groups: a datum is transformed by an unknown group element, and the goal is to recover an inverse transformation tha…
Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatism
Tianwei Ni, Esther Derman, Vineet Jain +3
Popular offline reinforcement learning (RL) methods rely on explicit conservatism, penalizing out-of-dataset actions or restricting rollout horizons. We question the universality o…
Energy Loss Functions for Physical Systems
Sékou-Oumar Kaba, Kusha Sareen, Daniel Levy +1
Effectively leveraging prior knowledge of a system's physics is crucial for applications of machine learning to scientific domains. Previous approaches mostly focused on incorporat…
Scaling Laws and Symmetry, Evidence from Neural Force Fields
Khang Ngo, Siamak Ravanbakhsh
We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a clear power-law scali…
Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
Behnoush Khavari, Mehran Shakerinava, Jayesh Khullar +4
Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ran…
Multi-Armed Sampling Problem and the End of Exploration
Mohammad Pedramfar, Siamak Ravanbakhsh
This paper introduces the framework of multi-armed sampling, which serves as the sampling counterpart to the optimization problem of multi-armed bandits. Our primary motivation is…
Diffusion Tree Sampling: Scalable inference-time alignment of diffusion models
Vineet Jain, Kusha Sareen, Mohammad Pedramfar +1
Adapting a pretrained diffusion model to new objectives at inference time remains an open problem in generative modeling. Existing steering methods suffer from inaccurate value est…
Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities
Tara Akhound-Sadegh, Jungyoon Lee, Avishek Joey Bose +7
Sampling efficiently from a target unnormalized probability density remains a core challenge, with relevance across countless high-impact scientific applications. A promising appro…
Beyond Scalar Rewards: An Axiomatic Framework for Lexicographic MDPs
Mehran Shakerinava, Siamak Ravanbakhsh, Adam Oberman
Recent work has formalized the reward hypothesis through the lens of expected utility theory, by interpreting reward as utility. Hausner's foundational work showed that dropping th…
On the Identifiability of Causal Abstractions
Xiusi Li, Sékou-Oumar Kaba, Siamak Ravanbakhsh
Causal representation learning (CRL) enhances machine learning models' robustness and generalizability by learning structural causal models associated with data-generating processe…
SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models
Daniel Levy, Siba Smarak Panigrahi, Sékou-Oumar Kaba +5
Generating novel crystalline materials has the potential to lead to advancements in fields such as electronics, energy storage, and catalysis. The defining characteristic of crysta…
Symmetry-Aware Generative Modeling through Learned Canonicalization
Kusha Sareen, Daniel Levy, Arnab Kumar Mondal +3
Generative modeling of symmetric densities has a range of applications in AI for science, from drug discovery to physics simulations. The existing generative modeling paradigm for…
Sampling from Energy-based Policies using Diffusion
Vineet Jain, Tara Akhound-Sadegh, Siamak Ravanbakhsh
Energy-based policies offer a flexible framework for modeling complex, multimodal behaviors in reinforcement learning (RL). In maximum entropy RL, the optimal policy is a Boltzmann…
E(3)-Equivariant Mesh Neural Networks
Thuan Trang, Nhat Khang Ngo, Daniel Levy +3
Triangular meshes are widely used to represent three-dimensional objects. As a result, many recent works have address the need for geometric deep learning on 3D mesh. However, we o…
Iterated Denoising Energy Matching for Sampling from Boltzmann Densities
Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose +9
Efficiently generating statistically independent samples from an unnormalized probability distribution, such as equilibrium samples of many-body systems, is a foundational problem…
Symmetry Breaking and Equivariant Neural Networks
Sékou-Oumar Kaba, Siamak Ravanbakhsh
Using symmetry as an inductive bias in deep learning has been proven to be a principled approach for sample-efficient model design. However, the relationship between symmetry and t…
Lie Point Symmetry and Physics Informed Networks
Tara Akhound-Sadegh, Laurence Perreault-Levasseur, Johannes Brandstetter +2
Symmetries have been leveraged to improve the generalization of neural networks through different mechanisms from data augmentation to equivariant architectures. However, despite t…
Weight-Sharing Regularization
Mehran Shakerinava, Motahareh Sohrabi, Siamak Ravanbakhsh +1
Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights of a neural network, defi…
Equivariant Adaptation of Large Pretrained Models
Arnab Kumar Mondal, Siba Smarak Panigrahi, Sékou-Oumar Kaba +2
Equivariant networks are specifically designed to ensure consistent behavior with respect to a set of input transformations, leading to higher sample efficiency and more accurate a…
Learning to Reach Goals via Diffusion
Vineet Jain, Siamak Ravanbakhsh
We present a novel perspective on goal-conditioned reinforcement learning by framing it within the context of denoising diffusion models. Analogous to the diffusion process, where…
Using Multiple Vector Channels Improves E(n)-Equivariant Graph Neural Networks
Daniel Levy, Sékou-Oumar Kaba, Carmelo Gonzales +2
We present a natural extension to E(n)-equivariant graph neural networks that uses multiple equivariant vectors per node. We formulate the extension and show that it improves perfo…