5 papers
Fusion of Graph Neural Networks via Optimal Transport
Weronika Ormaniec, Michael Vollenweider, Elisa Hoskovec
In this paper, we explore the idea of combining GCNs into one model. To that end, we align the weights of different models layer-wise using optimal transport (OT). We present and e…
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis
Weronika Ormaniec, Felix Dangel, Sidak Pal Singh
The Transformer architecture has inarguably revolutionized deep learning, overtaking classical architectures like multi-layer perceptrons (MLPs) and convolutional neural networks (…
Standardizing Structural Causal Models
Weronika Ormaniec, Scott Sussex, Lars Lorch +2
Synthetic datasets generated by structural causal models (SCMs) are commonly used for benchmarking causal structure learning algorithms. However, the variances and pairwise correla…
Position: Curvature Matrices Should Be Democratized via Linear Operators
Felix Dangel, Runa Eschenhagen, Weronika Ormaniec +3
Structured large matrices are prevalent in machine learning. A particularly important class is curvature matrices like the Hessian, which are central to understanding the loss land…
Transition Constrained Bayesian Optimization via Markov Decision Processes
Jose Pablo Folch, Calvin Tsay, Robert M Lee +6
Bayesian optimization is a methodology to optimize black-box functions. Traditionally, it focuses on the setting where you can arbitrarily query the search space. However, many rea…