5 citations · 5 across the 2 of their papers we have counts for
4 papers
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
WenHung Lee, Jian-Jia Chen, Xiaolin Lin +6
While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck whe…
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3
Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on…
AI Psychometrics: Evaluating the Psychological Reasoning of Large Language Models with Psychometric Validities
Yibai Li, Xiaolin Lin, Zhenghui Sha +2
The immense number of parameters and deep neural networks make large language models (LLMs) rival the complexity of human brains, which also makes them opaque ``black box'' systems…
CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3
Recent advances in large language models (LLMs) have significantly boosted long-context processing. However, the increasing key-value (KV) cache size poses critical challenges to m…