2 papers
cs.CL2025
Efficient Beam Search for Large Language Models Using Trie-Based Decoding
Brian J Chan, MaoXun Huang, Jui-Hung Cheng +2
This work presents a novel trie (prefix-tree)-based parallel decoding method that addresses the memory inefficiency of batch-based beam search. By sharing a single KV cache across…
cs.CL2025
Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks
Brian J Chan, Chao-Ting Chen, Jui-Hung Cheng +1
Retrieval-augmented generation (RAG) has gained traction as a powerful approach for enhancing language models by integrating external knowledge sources. However, RAG introduces cha…