132 citations · 524 across the 20 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2023★ 1 cited
REST: Retrieval-Based Speculative Decoding
Zhenyu He, Zexuan Zhong, Tianle Cai +2
We introduce Retrieval-Based Speculative Decoding (REST), a novel algorithm designed to speed up language model generation. The key insight driving the development of REST is the o…
cs.CL2023
Scaling In-Context Demonstrations with Structured Attention
Tianle Cai, Kaixuan Huang, Jason D. Lee +1
The recent surge of large language models (LLMs) highlights their ability to perform in-context learning, i.e., "learning" to perform a task from a few demonstrations in the contex…