most citedInstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference

1 citations · 1 across the 3 of their papers we have counts for

collaborators
Showing cs.ARShow all

2 papers · 1 filter

cs.AR2024

Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM

Zhongkai Yu, Shengwen Liang, Tianyun Ma +12

Deploying advanced large language models on edge devices, such as smartphones and robotics, is a growing trend that enhances user data privacy and network connectivity resilience w…

cs.AR2024★ 1 cited

InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference

Xiurui Pan, Endian Li, Qiao Li +6

The widespread of Large Language Models (LLMs) marks a significant milestone in generative AI. Nevertheless, the increasing context length and batch size in offline LLM inference e…