Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection

Jongseok Park, Sunga Kim, Alvin Cheung +1

Despite their importance in model sampling, efficient implementation of Top-k and Top-p algorithms for large vocabularies remains a significant challenge. Existing approaches often…