11 papers
UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge
Tianhao Jiang, Hang Gu, Teng Wang +9
Edge LLM inference combines sparsity and low-bit quantization to meet device memory, latency, and power limits. Yet quantization shrinks weight payloads without proportionally redu…
Realizable N:M Sparse Transformer Inference via Search-Kernel Co-Design
Yiming Liu, Wenqi Lou, Zhiguang Wang +4
The paper presents a co-designed hardware and software framework that enables fast inference of Vision Transformers by applying N:M structured sparsity with a specialized CUDA kern…
Scheduling Cause-Effect Chains without Timing Anomalies in End-to-End Latency
Yixuan Zhu, Bo Zhang, Yinkang Gao +7
In real-time systems, both individual task execution and data propagation must meet strict timing constraints. Cause-effect (CE) chains are widely used to analyze such behaviors by…
Hermes: A Unified High-Performance NTT Architecture with Hybrid Dataflow
Hang Gu, Teng Wang, Qianyu Cheng +6
Fully Homomorphic Encryption (FHE) relies heavily on the Number Theoretic Transform (NTT), making NTT a major performance bottleneck due to its intensive polynomial computations. H…
Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
Fengrui Zuo, Zhiwei Ke, Yiming Liu +3
Diffusion language models (DLMs) generate text through iterative denoising, but inference requires full-sequence attention at every iteration, resulting in substantial redundant co…
A Timing-Anomaly Free Dynamic Scheduling on Heterogeneous Systems
Yixuan Zhu, Yinkang Gao, Lei Gong +9
Heterogeneous systems commonly adopt dynamic scheduling algorithms to improve resource utilization and enhance scheduling flexibility. However, such flexibility may introduce timin…