Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches
Sunjoo Whang, Jungjun Oh, Minsung Kim +5
Long inputs and extended generation increase the storage and access costs of the key-value (KV) cache. Low-bit quantization reduces storage and memory traffic, while query-channel…
cs.LG2026
LLMForge: Multi-Backend Hardware-Aware Neural Architecture Search with Infinite-Head Attention for Edge Language Models
Xinting Jiang, Junyi Luo, Ruichen Qi +4
Sub-billion-parameter Transformer language models are increasingly deployed on edge devices, where the privacy, latency, and operating-cost advantages of on-device inference are co…