Realizable N:M Sparse Transformer Inference via Search-Kernel Co-Design

Yiming Liu, Wenqi Lou, Zhiguang Wang +4

Vision Transformers (ViTs) achieve strong accuracy but incur high inference latency. Semi-structured N:M sparsity can reduce arithmetic cost, yet its theoretical savings often fail…