1.5k citations · 1.7k across the 14 of their papers we have counts for
15 papers
TraceMem: Weaving Narrative Memory Schemata from User Conversational Traces
Yiming Shu, Pei Liu, Tiange Zhang +3
Sustaining long-term interactions remains a bottleneck for Large Language Models (LLMs), as their limited context windows struggle to manage dialogue histories that extend over tim…
ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution
Zican Dong, Peiyu Liu, Junyi Li +4
Recently, large language models (LLMs) have shown remarkable reasoning abilities by producing long reasoning traces. However, as the sequence length grows, the key-value (KV) cache…
FastV-RAG: Towards Fast and Fine-Grained Video QA with Retrieval-Augmented Generation
Gen Li, Peiyu Liu
Vision-Language Models (VLMs) excel at visual reasoning but still struggle with integrating external knowledge. Retrieval-Augmented Generation (RAG) is a promising solution, but cu…
How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
Han Peng, Peiyu Liu, Zican Dong +5
Diffusion language models (DLMs) have emerged as a promising alternative to the long-dominant autoregressive (AR) paradigm, offering a parallelable decoding process that could yiel…
Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
Zican Dong, Han Peng, Peiyu Liu +4
Mixture-of-Experts (MoE) models achieve a favorable trade-off between performance and inference efficiency by activating only a subset of experts. However, the memory overhead of s…
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
Peiyu Liu, Ze-Feng Gao, Wayne Xin Zhao +3
Key-value~(KV) caching is an important technique to accelerate the inference of large language models~(LLMs), but incurs significant memory overhead. To compress the size of KV cac…
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study
Peiyu Liu, Zikang Liu, Ze-Feng Gao +5
Despite the superior performance, Large Language Models~(LLMs) require significant computational resources for deployment and use. To overcome this issue, quantization methods have…
A semantically enhanced dual encoder for aspect sentiment triplet extraction
Baoxing Jiang, Shehui Liang, Peiyu Liu +2
Aspect sentiment triplet extraction (ASTE) is a crucial subtask of aspect-based sentiment analysis (ABSA) that aims to comprehensively identify sentiment triplets. Previous researc…
Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization
Tong Ye, Lingfei Wu, Tengfei Ma +5
Automatically generating human-readable text describing the functionality of a program is the intent of source code summarization. Although neural language models achieve significa…
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture
Peiyu Liu, Ze-Feng Gao, Yushuo Chen +2
In this paper, we propose a highly parameter-efficient approach to scaling pre-trained language models (PLMs) to a deeper model depth. Unlike prior work that shares all parameters…
A Survey of Large Language Models
Wayne Xin Zhao, Kun Zhou, Junyi Li +19
Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for compre…
TikTalk: A Video-Based Dialogue Dataset for Multi-Modal Chitchat in Real World
Hongpeng Lin, Ludan Ruan, Wenke Xia +8
To facilitate the research on intelligent and human-like chatbots with multi-modal context, we introduce a new video-based multi-modal dialogue dataset, called TikTalk. We collect…
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
Ze-Feng Gao, Peiyu Liu, Wayne Xin Zhao +2
Recently, Mixture-of-Experts (short as MoE) architecture has achieved remarkable success in increasing the model capacity of large-scale language models. However, MoE requires inco…
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators
Peiyu Liu, Ze-Feng Gao, Wayne Xin Zhao +3
This paper presents a novel pre-trained language models (PLM) compression approach based on the matrix product operator (short as MPO) from quantum many-body physics. It can decomp…
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
Yuqi Huo, Manli Zhang, Guangzhen Liu +32
Multi-modal pre-training models have been intensively explored to bridge vision and language in recent years. However, most of them explicitly model the cross-modal interaction bet…