5 papers
LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-design
Renjie Wei, Songqiang Xu, Linfeng Zhong +5
State space models (SSMs) like Mamba have recently attracted much attention. Compared to Transformer-based large language models (LLMs), Mamba achieves linear computation complexit…
SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding
Linfeng Zhong, Songqiang Xu, Huifeng Wen +4
The growing demand for efficient long-sequence modeling on edge devices has propelled widespread adoption of State Space Models (SSMs) like Mamba, due to their superior computation…
SpecASR: Accelerating LLM-based Automatic Speech Recognition via Speculative Decoding
Linye Wei, Shuzhang Zhong, Songqiang Xu +3
Large language model (LLM)-based automatic speech recognition (ASR) has recently attracted a lot of attention due to its high recognition accuracy and enhanced multi-dialect suppor…
FPQVAR: Floating Point Quantization for Visual Autoregressive Model with FPGA Hardware Co-design
Renjie Wei, Songqiang Xu, Qingyu Guo +1
Visual autoregressive (VAR) modeling has marked a paradigm shift in image generation from next-token prediction to next-scale prediction. VAR predicts a set of tokens at each step…
HG-PIPE: Vision Transformer Acceleration with Hybrid-Grained Pipeline
Qingyu Guo, Jiayong Wan, Songqiang Xu +2
Vision Transformer (ViT) acceleration with field programmable gate array (FPGA) is promising but challenging. Existing FPGA-based ViT accelerators mainly rely on temporal architect…