2 citations · 2 across the 2 of their papers we have counts for
4 papers
On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference
Siyu Ren, Kenny Q. Zhu
Despite the recent success associated with Large Language Models (LLMs), they are notably cost-prohibitive to deploy in resource-constrained environments due to their excessive mem…
Context Compression for Auto-regressive Transformers with Sentinel Tokens
Siyu Ren, Qi Jia, Kenny Q. Zhu
The quadratic complexity of the attention module makes it gradually become the bulk of compute in Transformer-based LLMs during generation. Moreover, the excessive key-value cache…
Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model
Qi Jia, Siyu Ren, Yizhu Liu +1
Despite tremendous improvements in natural language generation, summarization models still suffer from the unfaithfulness issue. Previous work evaluates faithfulness either using m…
EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling
Siyu Ren, Zhiyong Wu, Kenny Q. Zhu
Neural language models are probabilistic models of human text. They are predominantly trained using maximum likelihood estimation (MLE), which is equivalent to minimizing the forwa…