5 papers
When Large Language Models Meet Vector Databases: A Survey
Zhi Jing, Yongye Su, Yikun Han +5
This survey explores the synergistic potential of Large Language Models (LLMs) and Vector Databases (VecDBs), a burgeoning but rapidly evolving research area. With the proliferatio…
ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty
Meizhi Zhong, Xikai Liu, Chen Zhang +5
Large Language models (LLMs) have become a research hotspot. To accelerate the inference of LLMs, storing computed caches in memory has become the standard technique. However, as t…
Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
Meizhi Zhong, Chen Zhang, Yikun Lei +5
Enabling LLMs to handle lengthy context is currently a research hotspot. Most LLMs are built upon rotary position embedding (RoPE), a popular position encoding method. Therefore, a…
LLM with Relation Classifier for Document-Level Relation Extraction
Xingzuo Li, Kehai Chen, Yunfei Long +1
Large language models (LLMs) have created a new paradigm for natural language processing. Despite their advancement, LLM-based methods still lag behind traditional approaches in do…
MoDification: Mixture of Depths Made Easy
Chen Zhang, Meizhi Zhong, Qimeng Wang +8
Long-context efficiency has recently become a trending topic in serving large language models (LLMs). And mixture of depths (MoD) is proposed as a perfect fit to bring down both la…