15 papers
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6
The recent empirical success of the Muon optimizer has renewed interest in non-Euclidean optimization, typically justified by similarities with second-order methods, and linear min…
Adaptive Coordinate Transforms for Neural Operators
Chaoyu Liu, Zhonghao Li, Gaohang Chen +5
Neural operators have achieved promising performance on partial differential equations (PDEs), but most existing models are built on fixed Eulerian coordinates. This mismatch betwe…
Diffeomorphism-Equivariant Neural Networks
Josephine Elisabeth Oettinger, Zakhar Shumaylov, Johannes Bostelmann +2
Incorporating group symmetries via equivariance into neural networks has emerged as a robust approach for overcoming the efficiency and data demands of modern deep learning. While…
When is a System Discoverable from Data? Discovery Requires Chaos
Zakhar Shumaylov, Peter Zaika, Philipp Scholl +3
The deep learning revolution has spurred a rise in advances of using AI in sciences. Within physical sciences the main focus has been on discovery of dynamical systems from observa…
Learning Regularization Functionals for Inverse Problems: A Comparative Study
Johannes Hertrich, Hok Shing Wong, Alexander Denker +16
In recent years, a variety of learned regularization frameworks for solving inverse problems in imaging have emerged. These offer flexible modeling together with mathematical insig…
On Information Geometry and Iterative Optimization in Model Compression: Operator Factorization
Zakhar Shumaylov, Vasileios Tsiaras, Yannis Stylianou
The ever-increasing parameter counts of deep learning models necessitate effective compression techniques for deployment on resource-constrained devices. This paper explores the ap…