4 papers · 1 filter
Non-Uniform Parameter-Wise Model Merging
Albert Manuel Orozco Camacho, Stefan Horoi, Guy Wolf +1
Combining multiple machine learning models has long been a technique for enhancing performance, particularly in distributed settings. Traditional approaches, such as model ensemble…
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis
Stefan Horoi, Albert Manuel Orozco Camacho, Eugene Belilovsky +1
Combining the predictions of multiple trained models through ensembling is generally a good way to improve accuracy by leveraging the different learned features of the models, howe…
AdaFisher: Adaptive Second Order Optimization via Fisher Information
Damien Martins Gomes, Yanlei Zhang, Eugene Belilovsky +2
First-order optimization methods are currently the mainstream in training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by employing t…
Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation
Pedro Vianna, Muawiz Chaudhary, Paria Mehrbod +5
Deep neural networks have useful applications in many different tasks, however their performance can be severely affected by changes in the data distribution. For example, in the b…