1 citations · 6 across the 34 of their papers we have counts for
8 papers · 1 filter
Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
Pei-Shuo Wang, Jian-Jia Chen, Chun-Che Yang +4
The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strateg…
Regression Language Models for Code
Yash Akhauri, Xingyou Song, Arissa Wongpanich +2
We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods ha…
FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
Zhanqiu Hu, Jian Meng, Yash Akhauri +4
Diffusion language models offer parallel token generation and inherent bidirectionality, promising more efficient and powerful sequence modeling compared to autoregressive approach…
SplitReason: Learning To Offload Reasoning
Yash Akhauri, Anthony Fei, Chi-Chih Chang +3
Reasoning in large language models (LLMs) tends to produce substantially longer token generation sequences than simpler language modeling tasks. This extended generation length ref…
xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin +8
Long-context Large Language Models (LLMs) enable powerful applications but incur high memory costs due to the key-value states (KV-Cache). Recent studies attempt to share KV-Cache…
TokenButler: Token Importance is Predictable
Yash Akhauri, Ahmed F AbouElhamayed, Yifei Gao +4
Large Language Models (LLMs) rely on the Key-Value (KV) Cache to store token history, enabling efficient decoding of tokens. As the KV-Cache grows, it becomes a major memory and co…
NITRO: LLM Inference on Intel Laptop NPUs
Anthony Fei, Mohamed S. Abdelfattah
Large Language Models (LLMs) have become essential tools in natural language processing, finding large usage in chatbots such as ChatGPT and Gemini, and are a central area of resea…
Radial Networks: Dynamic Layer Routing for High-Performance Large Language Models
Jordan Dotzel, Yash Akhauri, Ahmed S. AbouElhamayed +3
Large language models (LLMs) often struggle with strict memory, latency, and power demands. To meet these demands, various forms of dynamic sparsity have been proposed that reduce…