4 papers
SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Coding
Jiamu Zhang, Liang Wu, Kelly Wan +2
Large language models (LLMs) increasingly read long inputs in the agentic era, from whole documents and codebases to conversations across many turns. Their inference memory is then…
WiSP: A Working-Set View of Mixture-of-Experts Serving on Extremely Low-Resource Hardware
Jiamu Zhang, Liang Wu, Mayank Darbari +1
Modern Mixture-of-Experts (MoE) models place most of their parameters in expert layers, yet only a small fraction of those experts are used for any token. The unused weights must s…
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Yang Sui, Yu-Neng Chuang, Guanchu Wang +9
Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as OpenAI o1 and DeepSeek-R1, ha…
The Science of Evaluating Foundation Models
Jiayi Yuan, Jiamu Zhang, Andrew Wen +1
The emergent phenomena of large foundation models have revolutionized natural language processing. However, evaluating these models presents significant challenges due to their siz…