1 citations · 2 across the 7 of their papers we have counts for
5 papers · 1 filter
OVAL: Output-Aware Local Page Bases for KV Cache Retrieval
Ashkan Shahbazi, Chayne Thrash, Soheil Kolouri
Long context inference with large language models becomes increasingly expensive as attention must operate over an ever growing KV cache. Page sparse attention reduces this cost by…
Output-aware Residual Stream Pruning for Large Language Models
Chayne Thrash, Kevin Chen, Soheil Kolouri
Residual stream pruning methods reduce inference cost by shrinking the model's hidden dimension, but existing approaches typically choose these dimensions by minimizing activation…
IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
Ali Abbasi, Chayne Thrash, Haoran Qin +2
Large language models deliver strong performance across language and reasoning tasks, but their storage and compute costs remain major barriers to deployment in resource-constraine…
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
Chayne Thrash, Ali Abbasi, Soheil Kolouri
Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost. Weight-activation quantization can reduce these costs, but low-bit ac…
Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression
Ali Abbasi, Chayne Thrash, Haoran Qin +3
Advances in large language models have driven strong performance across many tasks, but their memory and compute costs still hinder deployment. SVD-based compression reduces storag…