Leveraging Compute-in-Memory for Efficient Generative Model Inference in TPUs

Zhantong Zhu, Hongou Li, Wenjie Ren +4

With the rapid advent of generative models, efficiently deploying these models on specialized hardware has become critical. Tensor Processing Units (TPUs) are designed to accelerat…