50 citations · 73 across the 14 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.DC2024★ 1 cited
KVDirect: Distributed Disaggregated LLM Inference
Shiyang Chen, Rain Jiang, Dezhi Yu +6
Large Language Models (LLMs) have become the new foundation for many applications, reshaping human society like a storm. Disaggregated inference, which separates prefill and decode…
cs.CR2024
PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips
Zachary Coalson, Jeonghyun Woo, Chris S. Lin +8
We study a new vulnerability in commercial-scale safety-aligned large language models (LLMs): their refusal to generate harmful responses can be broken by flipping only a few bits…
cs.LG2024★ 3 cited
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
Haojun Xia, Zhen Zheng, Xiaoxia Wu +10
Six-bit quantization (FP6) can effectively reduce the size of large language models (LLMs) and preserve the model quality consistently across varied applications. However, existing…