7 papers
Depth Exploration for LLM Decoding
Weisi Yang, Zipeng Sun, Stephen Xia
Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth…
Attention Drift: What Autoregressive Speculative Decoding Models Learn
DoÄaç Eldenk, Payal Mohapatra, Yigitcan Comlek +3
Speculative decoding accelerates LLM inference by drafting future tokens with a small model, but drafter models degrade sharply under template perturbation and long-context inputs.…
Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
Yueyuan Sui, Payal Mohapatra, DoÄaç Eldenk +5
Edge devices increasingly run multimodal sensing pipelines that must remain accurate despite fluctuating power budgets and unpredictable sensor dropout. Existing pruning methods fa…
UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models
DoÄaç Eldenk, Stephen Xia
Developing and evaluating distributed inference algorithms remains difficult due to the lack of standardized tools for modeling heterogeneous devices and networks. Existing studies…
MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series
Payal Mohapatra, Yueyuan Sui, Akash Pandey +2
From clinical healthcare to daily living, continuous sensor monitoring across multiple modalities has shown great promise for real-world intelligent decision-making but also faces…
MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs
Hongjun Xu, Junxi Xia, Weisi Yang +2
Deploying Mamba models on microcontrollers (MCUs) remains challenging due to limited memory, the lack of native operator support, and the absence of embedded-friendly toolchains. W…