6 papers · 1 filter
Depth Exploration for LLM Decoding
Weisi Yang, Zipeng Sun, Stephen Xia
Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth…
Attention Drift: What Autoregressive Speculative Decoding Models Learn
DoÄaç Eldenk, Payal Mohapatra, Yigitcan Comlek +3
Speculative decoding accelerates LLM inference by drafting future tokens with a small model, but drafter models degrade sharply under template perturbation and long-context inputs.…
Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
Yueyuan Sui, Payal Mohapatra, DoÄaç Eldenk +5
Edge devices increasingly run multimodal sensing pipelines that must remain accurate despite fluctuating power budgets and unpredictable sensor dropout. Existing pruning methods fa…
MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series
Payal Mohapatra, Yueyuan Sui, Akash Pandey +2
From clinical healthcare to daily living, continuous sensor monitoring across multiple modalities has shown great promise for real-world intelligent decision-making but also faces…
MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs
Hongjun Xu, Junxi Xia, Weisi Yang +2
Deploying Mamba models on microcontrollers (MCUs) remains challenging due to limited memory, the lack of native operator support, and the absence of embedded-friendly toolchains. W…
ElastiFormer: Learned Redundancy Reduction in Transformer via Self-Distillation
Junzhang Liu, Tingkai Liu, Yueyuan Sui +1
We introduce ElastiFormer, a post-training technique that adapts pretrained Transformer models into an elastic counterpart with variable inference time compute. ElastiFormer introd…