4 papers
PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling
Weisi Yang, Stephen Xia
Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased privacy, personalization, and red…
Depth Exploration for LLM Decoding
Weisi Yang, Zipeng Sun, Stephen Xia
Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth…
MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs
Hongjun Xu, Junxi Xia, Weisi Yang +2
Deploying Mamba models on microcontrollers (MCUs) remains challenging due to limited memory, the lack of native operator support, and the absence of embedded-friendly toolchains. W…
Towards On-Device Evidence Gathering for Intimate Partner Infiltration: A Feasibility Study for Joint Identity-Action Detection
Weisi Yang, Shinan Liu, Feng Xiao +2
Intimate Partner Infiltration (IPI) refers to phone-side privacy infiltration in intimate or close relationships, often enabled by physical access to a person's smartphone and disc…