9 papers · 1 filter
Designing Spatial Architectures for Sparse Attention: STAR Accelerator via Cross-Stage Tiling
Huizheng Wang, Taiquan Wei, Hongbin Wang +6
Large language models (LLMs) rely on self-attention for contextual understanding, demanding high-throughput inference and large-scale token parallelism (LTPP). Existing dynamic spa…
cuPilot: A Strategy-Coordinated Multi-agent Framework for CUDA Kernel Evolution
Jinwu Chen, Qidie Wu, Bin Li +5
Optimizing CUDA kernels is a challenging and labor-intensive task, given the need for hardware-software co-design expertise and the proprietary nature of high-performance kernel li…
TEMP: A Memory Efficient Physical-aware Tensor Partition-Mapping Framework on Wafer-scale Chips
Huizheng Wang, Taiquan Wei, Zichuan Wang +8
Large language models (LLMs) demand significant memory and computation resources. Wafer-scale chips (WSCs) provide high computation power and die-to-die (D2D) bandwidth but face a…
WATOS: Efficient LLM Training Strategies and Architecture Co-exploration for Wafer-scale Chip
Huizheng Wang, Zichuan Wang, Hongbin Wang +5
Training large language models (LLMs) imposes extreme demands on computation, memory capacity, and interconnect bandwidth, driven by their ever-increasing parameter scales and inte…
BitStopper: An Efficient Transformer Attention Accelerator via Stage-fusion and Early Termination
Huizheng Wang, Hongbin Wang, Shaojun Wei +2
Attention-based large language models (LLMs) have transformed modern AI applications, but the quadratic cost of self-attention imposes significant compute and memory overhead. Dyna…
LAPA: Log-Domain Prediction-Driven Dynamic Sparsity Accelerator for Transformer Model
Huizheng Wang, Hongbin Wang, Shaojun Wei +2
Attention-based Transformers have revolutionized natural language processing (NLP) and shown strong performance in computer vision (CV) tasks. However, as the input sequence varies…
MoEntwine: Unleashing the Potential of Wafer-scale Chips for Large-scale Expert Parallel Inference
Xinru Tang, Jingxiang Hou, Dingcheng Jiang +9
As large language models (LLMs) continue to scale up, mixture-of-experts (MoE) has become a common technology in SOTA models. MoE models rely on expert parallelism (EP) to alleviat…
MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
Huizheng Wang, Zichuan Wang, Zhiheng Yue +8
Large language models (LLMs) face significant inference latency due to inefficiencies in GEMM operations, weight access, and KV cache access, especially in real-time scenarios. Thi…
Hexcute: A Compiler Framework for Automating Layout Synthesis in GPU Programs
Xiao Zhang, Yaoyao Ding, Bolin Sun +3
Efficient GPU programming is crucial for achieving high performance in deep learning (DL) applications. The performance of GPU programs depends on how data is parallelized across t…