37 citations · 81 across the 14 of their papers we have counts for
14 papers
A Model Zoo of Vision Transformers
Damian Falk, Léo Meynent, Florence Pfammatter +2
The availability of large, structured populations of neural networks - called 'model zoos' - has led to the development of a multitude of downstream tasks ranging from model analys…
The Impact of Model Zoo Size and Composition on Weight Space Learning
Damian Falk, Konstantin Schürholt, Damian Borth
Re-using trained neural network models is a common strategy to reduce training cost and transfer knowledge. Weight space learning - using the weights of trained models as data moda…
Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction
Léo Meynent, Ivan Melev, Konstantin Schürholt +2
The weights of neural networks (NNs) have recently gained prominence as a new data modality in machine learning, with applications ranging from accuracy and hyperparameter predicti…
Hyper-Representations: Learning from Populations of Neural Networks
Konstantin Schürholt
This thesis addresses the challenge of understanding Neural Networks through the lens of their most fundamental component: the weights, which encapsulate the learned information an…
Towards Scalable and Versatile Weight Space Learning
Konstantin Schürholt, Michael W. Mahoney, Damian Borth
Learning representations of well-trained neural network models holds the promise to provide an understanding of the inner workings of those models. However, previous work has eithe…
Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models
Dominik Honegger, Konstantin Schürholt, Damian Borth
With growing size of Neural Networks (NNs), model sparsification to reduce the computational cost and memory demand for model inference has become of vital interest for both resear…
Towards dynamic stability analysis of sustainable power grids using graph neural networks
Christian Nauck, Michael Lindner, Konstantin Schürholt +1
To mitigate climate change, the share of renewable needs to be increased. Renewable energies introduce new challenges to power grids due to decentralization, reduced inertia and vo…
Model Zoos: A Dataset of Diverse Populations of Neural Network Models
Konstantin Schürholt, Diyar Taskiran, Boris Knyazev +2
In the last years, neural networks (NN) have evolved from laboratory environments to the state-of-the-art for many real-world problems. It was shown that NN models (i.e., their wei…
Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights
Konstantin Schürholt, Boris Knyazev, Xavier Giró-i-Nieto +1
Learning representations of neural network weights given a model zoo is an emerging and challenging area with many potential applications from model inspection, to neural architect…
Hyper-Representations for Pre-Training and Transfer Learning
Konstantin Schürholt, Boris Knyazev, Xavier Giró-i-Nieto +1
Learning representations of neural network weights given a model zoo is an emerging and challenging area with many potential applications from model inspection, to neural architect…
Toward Dynamic Stability Assessment of Power Grid Topologies using Graph Neural Networks
Christian Nauck, Michael Lindner, Konstantin Schürholt +1
To mitigate climate change, the share of renewable energies in power production needs to be increased. Renewables introduce new challenges to power grids regarding the dynamic stab…
Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction
Konstantin Schürholt, Dimche Kostadinov, Damian Borth
Self-Supervised Learning (SSL) has been shown to learn useful and information-preserving representations. Neural Networks (NNs) are widely applied, yet their weight space is still…
Predicting Basin Stability of Power Grids using Graph Neural Networks
Christian Nauck, Michael Lindner, Konstantin Schürholt +5
The prediction of dynamical stability of power grids becomes more important and challenging with increasing shares of renewable energy sources due to their decentralized structure,…
An Investigation of the Weight Space to Monitor the Training Progress of Neural Networks
Konstantin Schürholt, Damian Borth
Safe use of Deep Neural Networks (DNNs) requires careful testing. However, deployed models are often trained further to improve in performance. As rigorous testing and evaluation i…