3 citations · 12 across the 16 of their papers we have counts for
17 papers
Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks
Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara +1
In this short note we consider the gradient descent dynamics of deep scalar linear networks, , which enjoy exact time-course solutions for any integer d…
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2
Recent frontier large language models predominantly rely on Mixture-of-Experts (MoE) architectures. Despite empirical progress, there is still no principled understanding of how hy…
Training Neural Networks at Any Scale
Thomas Pethick, Kimon Antonakopoulos, Antonio Silveti-Falls +2
This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a…
The Amazon Nova Family of Models: Technical Report and Model Card
Amazon AGI, Aaron Langford, Aayush Shah +783
We present Amazon Nova, a new generation of state-of-the-art foundation models that deliver frontier intelligence and industry-leading price performance. Amazon Nova Pro is a highl…
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
Moritz Haas, Sebastian Bordt, Ulrike von Luxburg +1
Scaling limits, such as infinite-width limits, serve as promising theoretical tools to study large-scale models. However, it is widely believed that existing infinite-width theory…
μP: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
Moritz Haas, Jin Xu, Volkan Cevher +1
Sharpness Aware Minimization (SAM) enhances performance across various neural architectures and datasets. As models are continually scaled up to improve performance, a rigorous und…
Explaining Kernel Clustering via Decision Trees
Maximilian Fleissner, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Despite the growing popularity of explainable and interpretable machine learning, there is still surprisingly limited work on inherently interpretable clustering methods. Recently,…
Self-Compatibility: Evaluating Causal Discovery without Ground Truth
Philipp M. Faller, Leena Chennuru Vankadara, Atalanti A. Mastakouri +2
As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data. This is concerning, given that simulations reflect preconcepti…
Reinterpreting causal discovery as the task of predicting unobserved joint statistics
Dominik Janzing, Philipp M. Faller, Leena Chennuru Vankadara
If denote sets of random variables, two different data sources may contain samples from and , respectively. We argue that causal discovery can help infer…
A Consistent Estimator for Confounding Strength
Luca Rendsburg, Leena Chennuru Vankadara, Debarghya Ghoshdastidar +1
Regression on observational data can fail to capture a causal relationship in the presence of unobserved confounding. Confounding strength measures this mismatch, but estimating it…
Interpolation and Regularization for Causal Learning
Leena Chennuru Vankadara, Luca Rendsburg, Ulrike von Luxburg +1
We study the problem of learning causal models from observational data through the lens of interpolation and its counterpart -- regularization. A large volume of recent theoretical…
Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks
Pascal Mattia Esser, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
In recent years, several results in the supervised learning setting suggested that classical statistical learning-theoretic measures, such as VC dimension, do not adequately explai…
Graphon based Clustering and Testing of Networks: Algorithms and Theory
Mahalakshmi Sabanayagam, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Network-valued data are encountered in a wide range of applications and pose challenges in learning due to their complex structure and absence of vertex correspondence. Typical exa…
Causal Forecasting:Generalization Bounds for Autoregressive Models
Leena Chennuru Vankadara, Philipp Michael Faller, Michaela Hardt +3
Despite the increasing relevance of forecasting methods, causal implications of these algorithms remain largely unexplored. This is concerning considering that, even under simplify…
Recovery Guarantees for Kernel-based Clustering under Non-parametric Mixture Models
Leena Chennuru Vankadara, Sebastian Bordt, Ulrike von Luxburg +1
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation pr…
Insights into Ordinal Embedding Algorithms: A Systematic Evaluation
Leena Chennuru Vankadara, Siavash Haghiri, Michael Lohaus +2
The objective of ordinal embedding is to find a Euclidean representation of a set of abstract items, using only answers to triplet comparisons of the form "Is item closer to th…
On the optimality of kernels for high-dimensional clustering
Leena Chennuru Vankadara, Debarghya Ghoshdastidar
This paper studies the optimality of kernel methods in high-dimensional data clustering. Recent works have studied the large sample performance of kernel clustering in the high-dim…