OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference

Seungjun Shin, Jaehoon Oh, Dokwan Oh

Attention mechanisms are central to the success of large language models (LLMs), enabling them to capture intricate token dependencies and implicitly assign importance to each toke…