216 citations · 454 across the 55 of their papers we have counts for
10 papers · 1 filter
Markov Equivalence and Consistency in Differentiable Structure Learning
Chang Deng, Kevin Bello, Pradeep Ravikumar +1
Existing approaches to differentiable structure learning of directed acyclic graphs (DAGs) rely on strong identifiability assumptions in order to guarantee that global minimizers o…
Dimension-independent rates for structured neural density estimation
Robert A. Vandermeulen, Wai Ming Tai, Bryon Aragam
We show that deep neural networks achieve dimension-independent rates of convergence for learning structured densities such as those arising in image, audio, video, and text applic…
Identifying General Mechanism Shifts in Linear Causal Representations
Tianyu Chen, Kevin Bello, Francesco Locatello +2
We consider the linear causal representation learning setting where we observe a linear mixing of unknown latent factors, which follow a linear structural causal model. Recent…
Breaking the curse of dimensionality in structured density estimation
Robert A. Vandermeulen, Wai Ming Tai, Bryon Aragam
We consider the problem of estimating a structured multivariate density, subject to Markov conditions implied by an undirected graph. In the worst case, without Markovian assumptio…
Model-free Estimation of Latent Structure via Multiscale Nonparametric Maximum Likelihood
Bryon Aragam, Ruiyi Yang
Multivariate distributions often carry latent structures that are difficult to identify and estimate, and which better reflect the data generating mechanism than extrinsic structur…
Greedy equivalence search for nonparametric graphical models
Bryon Aragam
One of the hallmark achievements of the theory of graphical models and Bayesian model selection is the celebrated greedy equivalence search (GES) algorithm due to Chickering and Me…
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +1
Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be eas…
On the Origins of Linear Representations in Large Language Models
Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar +2
Recent works have argued that high-level semantic concepts are encoded "linearly" in the representation space of large language models. In this work, we study the origins of such l…
Optimal estimation of Gaussian (poly)trees
Yuhao Wang, Ming Gao, Wai Ming Tai +2
We develop optimal algorithms for learning undirected Gaussian trees and directed Gaussian polytrees from data. We consider both problems of distribution learning (i.e. in KL dista…
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
Goutham Rajendran, Simon Buchholz, Bryon Aragam +2
To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causa…