59 citations · 59 across the 13 of their papers we have counts for
14 papers
Rethinking Unified Memory for NPU-PIM Systems: Dual-View Memory for Dynamic Inference of LLM
Shixin Zhao, Lian Liu, Tianhua Han +3
Heterogeneous architectures that combine neural processing unit (NPU) and processing-in-memory (PIM) are increasingly adopted to accelerate LLM inference. Prior work focuses on bui…
Ouroboros: Wafer-Scale SRAM CIM with Token-Grained Pipelining for Large Language Model Inference
Yiqi Liu, Yudong Pan, Mengdi Wang +5
Conventional LLM inference architectures suffer from high energy and latency due to frequent data movement across memory hierarchies. We propose Ouroboros, a wafer-scale SRAM-based…
TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
Yudong Pan, Yintao He, Tianhua Han +7
To deploy large Mixture-of-Experts (MoE) models cost-effectively, offloading-based single-GPU heterogeneous inference is crucial. While GPU-CPU architectures that offload cold expe…
PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
Lian Liu, Shixin Zhao, Yutian Zhou +4
The widespread adoption of Large Language Models (LLMs) has exponentially increased the demand for efficient serving systems. With growing requests and context lengths, key-value (…
From Buffers to Registers: Unlocking Fine-Grained FlashAttention with Hybrid-Bonded 3D NPU Co-Design
Jinxin Yu, Yudong Pan, Mengdi Wang +4
Transformer-based models dominate modern AI workloads but exacerbate memory bottlenecks due to their quadratic attention complexity and ever-growing model sizes. Existing accelerat…
Lifecycle Cost-Effectiveness Modeling for Redundancy-Enhanced Multi-Chiplet Architectures
Zizhen Liu, Fangzhiyi Wang, Mengdi Wang +4
The growing demand for compute-intensive applications has made multi-chiplet architectures a promising alternative to monolithic designs, offering improved scalability and manufact…
CLASS: A Controller-Centric Layout Synthesizer for Dynamic Quantum Circuits
Yu Chen, Yilun Zhao, Bing Li +4
Layout Synthesis for Quantum Computing (LSQC) is a critical component of quantum design tools. Traditional LSQC studies primarily focus on optimizing for reduced circuit depth by a…
ChipSeek: Optimizing Verilog Generation via EDA-Integrated Reinforcement Learning
Zhirong Chen, Kaiyan Chang, Zhuolin Li +8
Large Language Models have emerged as powerful tools for automating Register-Transfer Level (RTL) code generation, yet they face critical limitations: existing approaches typically…
Be CIM or Be Memory: A Dual-mode-aware DNN Compiler for CIM Accelerators
Shixin Zhao, Yuming Li, Bing Li +4
Computing-in-memory (CIM) architectures demonstrate superior performance over traditional architectures. To unleash the potential of CIM accelerators, many compilation methods have…
Make LLM Inference Affordable to Everyone: Augmenting GPU Memory with NDP-DIMM
Lian Liu, Shixin Zhao, Bing Li +6
The billion-scale Large Language Models (LLMs) need deployment on expensive server-grade GPUs with large-storage HBMs and abundant computation capability. As LLM-assisted services…
RTLMarker: Protecting LLM-Generated RTL Copyright via a Hardware Watermarking Framework
Kun Wang, Kaiyan Chang, Mengdi Wang +4
Recent advances of large language models in the field of Verilog generation have raised several ethical and security concerns, such as code copyright protection and dissemination o…
COMET: Towards Partical W4A4KV4 LLMs Serving
Lian Liu, Haimeng Ren, Long Cheng +6
Quantization is a widely-used compression technology to reduce the overhead of serving large language models (LLMs) on terminal devices and in cloud data centers. However, prevalen…
Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation
Kaiyan Chang, Zhirong Chen, Yunhao Zhou +9
Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large la…
Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework
Kaiyan Chang, Kun Wang, Nan Yang +16
Recent advances in large language models have demonstrated their potential for automated generation of hardware description language (HDL) code from high-level prompts. Researchers…