Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition

Yang Li, Liangzhen Lai, Yuan Shangguan +5

Transformer-based models excel in speech recognition. Existing efforts to optimize Transformer inference, typically for long-context applications, center on simplifying attention s…