Operator Fusion for LLM Inference on the Tensix Architecture

Qingbo Wu, Ke Li, Wenzhu Wang +3

This study addresses on-device inference bottlenecks of Transformer models on Tenstorrent's Tensix architecture and proposes an operator fusion strategy that enhances data locality…