Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
H1B-KV: Hybrid One-Bit Caches for Memory-Efficient Large Language Model Inference
Harshil Vejendla
Autoregressive decoding in large language models (LLMs) requires caching a growing list of past key-value (KV) pairs, making long-context inference a memory-bound problem. While re…
cs.CL2025
SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
Harshil Vejendla
Mixture-of-Experts (MoE) layers scale transformers by routing tokens to a sparse subset of feed-forward experts. Token-level routing, however, assigns an entire semantic spectrum t…