5 papers · 1 filter
HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference
Xuan Ai, Qingqing Yang, Peng Wang +4
Long-context inference in Large Language Models (LLMs) is bottlenecked by the quadratic computation complexity of attention and the substantial memory footprint of Key-Value (KV) c…
AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM
Haoyu Huang, Hong Ting Tsang, Jiaxin Bai +3
Retrieval-augmented generation (RAG) has shown some success in augmenting large language models (LLMs) with external knowledge. However, as a non-parametric knowledge integration p…
CentroidKV: Efficient Long-Context LLM Inference via KV Cache Clustering
Jie Hu, Shengnan Wang, Yutong He +8
Large language models (LLMs) with extended context windows have become increasingly prevalent for tackling complex tasks. However, the substantial Key-Value (KV) cache required for…
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora
Jiaxin Bai, Wei Fan, Qi Hu +17
We present AutoSchemaKG, a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas. Our system leverages large language models t…
XL3M: A Training-free Framework for LLM Length Extension Based on Segment-wise Inference
Shengnan Wang, Youhui Bai, Lin Zhang +7
Length generalization failure problem, namely the large language model (LLM) fails to generalize to texts longer than its maximum training length, greatly restricts the application…