3 papers
cs.LG2026
ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits
Aryan Karmore
In current Mixture of Experts (MoE) architectures, linear memory scaling is present, the memory grows as the number of experts increases. independent expert weight matrices req…
cs.CV2026
ButterflyViT: 354 Expert Compression for Edge Vision Transformers
Aryan Karmore
Deploying sparse Mixture of Experts(MoE) Vision Transformers remains a challenge due to linear expert memory scaling. Linear memory scaling stores independent expert weight mat…
cs.LG2026
LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers
Aryan Karmore
Compressing the KV cache is a required step to deploy large language models on edge devices. Current quantization methods compress storage but fail to reduce bandwidth as attention…