Token-Operations-Oriented Inference Optimization Techniques for Large Models

Shiguo Lian, Kai Wang, Zhaoxiang Liu +23

Large model inference optimization serves as a key foundation for supporting the scalable, low-cost, and highly stable operation of large model services. Centered on token-oriented…