Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inference

Claudio Angione, Yue Zhao, Harry Yang +4

The rapid growth of large-scale AI models, particularly large language models has brought significant challenges in data privacy, computational resources, and accessibility. Tradit…