Majority Kernels: An Approach to Leverage Big Model Dynamics for Efficient Small Model Training

Hanna Mazzawi, Pranjal Awasthi, Xavi Gonzalvo +1

Recent breakthroughs and successful deployment of large language and vision models in a constrained environment predominantly follow a two phase approach. First, large models are t…