Learning Modular Structures That Generalize Out-of-Distribution
arXiv:2208.03753 · doi:10.1609/aaai.v36i11.21589
Abstract
Out-of-distribution (O.O.D.) generalization remains to be a key challenge for real-world machine learning systems. We describe a method for O.O.D. generalization that, through training, encourages models to only preserve features in the network that are well reused across multiple training domains. Our method combines two complementary neuron-level regularizers with a probabilistic differentiable binary mask over the network, to extract a modular sub-network that achieves better O.O.D. performance than the original network. Preliminary evaluation on two benchmark datasets corroborates the promise of our method.
Accepted at AAAI 2022 Student Abstract and Poster Program