Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator
YuXin Li, Felix Dangel, Derek Tam +1
The diagonal of a model's Fisher Information Matrix (the "Fisher diagonal") has frequently been used as a way to measure parameter sensitivity. Typically, the Fisher diagonal is es…
cs.LG2024
Realistic Evaluation of Model Merging for Compositional Generalization
Derek Tam, Yash Kant, Brian Lester +2
Merging has become a widespread way to cheaply combine individual models into a single model that inherits their capabilities and attains better performance. This popularity has sp…
cs.LG2024
Merging by Matching Models in Task Parameter Subspaces
Derek Tam, Mohit Bansal, Colin Raffel
Model merging aims to cheaply combine individual task-specific models into a single multitask model. In this work, we view past merging methods as leveraging different notions of a…