Compressing Models with Few Samples: Mimicking then Replacing

Huanyu Wang, Junjie Liu, Xin Ma +3

Few-sample compression aims to compress a big redundant model into a small compact one with only few samples. If we fine-tune models with these limited few samples directly, models…