4 papers · 1 filter
More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
Jiebin Zhang, Dawei Zhu, Yifan Song +6
As large language models (LLMs) process increasing context windows, the memory usage of KV cache has become a critical bottleneck during inference. The mainstream KV compression me…
Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
Xuming Hu, Xiaochuan Li, Junzhe Chen +8
Generative search engines have the potential to transform how people seek information online, but generated responses from existing large language models (LLMs)-backed generative s…
Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer
Xinshuo Hu, Baotian Hu, Dongfang Li +2
The present study introduces the knowledge-augmented generator, which is specifically designed to produce information that remains grounded in contextual knowledge, regardless of a…
PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models
Haochen Tan, Zhijiang Guo, Zhan Shi +8
Large Language Models (LLMs) have succeeded remarkably in understanding long-form contents. However, exploring their capability for generating long-form contents, such as reports a…