3 citations · 12 across the 16 of their papers we have counts for
11 papers · 1 filter
Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks
Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara +1
In this short note we consider the gradient descent dynamics of deep scalar linear networks, , which enjoy exact time-course solutions for any integer d…
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2
Recent frontier large language models predominantly rely on Mixture-of-Experts (MoE) architectures. Despite empirical progress, there is still no principled understanding of how hy…
Training Neural Networks at Any Scale
Thomas Pethick, Kimon Antonakopoulos, Antonio Silveti-Falls +2
This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a…
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
Moritz Haas, Sebastian Bordt, Ulrike von Luxburg +1
Scaling limits, such as infinite-width limits, serve as promising theoretical tools to study large-scale models. However, it is widely believed that existing infinite-width theory…
μP: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
Moritz Haas, Jin Xu, Volkan Cevher +1
Sharpness Aware Minimization (SAM) enhances performance across various neural architectures and datasets. As models are continually scaled up to improve performance, a rigorous und…
Explaining Kernel Clustering via Decision Trees
Maximilian Fleissner, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Despite the growing popularity of explainable and interpretable machine learning, there is still surprisingly limited work on inherently interpretable clustering methods. Recently,…
Self-Compatibility: Evaluating Causal Discovery without Ground Truth
Philipp M. Faller, Leena Chennuru Vankadara, Atalanti A. Mastakouri +2
As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data. This is concerning, given that simulations reflect preconcepti…
Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks
Pascal Mattia Esser, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
In recent years, several results in the supervised learning setting suggested that classical statistical learning-theoretic measures, such as VC dimension, do not adequately explai…
Graphon based Clustering and Testing of Networks: Algorithms and Theory
Mahalakshmi Sabanayagam, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Network-valued data are encountered in a wide range of applications and pose challenges in learning due to their complex structure and absence of vertex correspondence. Typical exa…
Recovery Guarantees for Kernel-based Clustering under Non-parametric Mixture Models
Leena Chennuru Vankadara, Sebastian Bordt, Ulrike von Luxburg +1
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation pr…
Insights into Ordinal Embedding Algorithms: A Systematic Evaluation
Leena Chennuru Vankadara, Siavash Haghiri, Michael Lohaus +2
The objective of ordinal embedding is to find a Euclidean representation of a set of abstract items, using only answers to triplet comparisons of the form "Is item closer to th…