Memorizing Long-tail Data Can Help Generalization Through Composition

Mo Zhou, Haoyang Ma, Rong Ge

Deep learning has led researchers to rethink the relationship between memorization and generalization. In many settings, memorization does not hurt generalization due to implicit r…