1 citations · 2 across the 12 of their papers we have counts for
12 papers
CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language
Qi Fan, An Zou, Yehan Ma
Developing high-performance CUDA kernels demands specialized knowledge in algorithm implementation, correctness validation, and hardware-aware parallel optimization, creating a sub…
River-LLM: Large Language Model Seamless Exit Based on KV Share
Yingtao Shen, An Zou
Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency. Early Exit has emerged as…
CUDABench: Benchmarking LLMs for Text-to-CUDA Generation
Jiace Zhu, Wentao Chen, Qi Fan +6
Recent studies have demonstrated the potential of Large Language Models (LLMs) in generating GPU Kernels. Current benchmarks focus on the translation of high-level languages into C…
TimeBill: Time-Budgeted Inference for Large Language Models
Qi Fan, An Zou, Yehan Ma
Large Language Models (LLMs) are increasingly deployed in time-critical systems, such as robotics, autonomous driving, embodied intelligence, and industrial automation, where gener…
CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
Wentao Chen, Jiace Zhu, Qi Fan +2
Large Language Models (LLMs) have demonstrated strong capabilities in general-purpose code generation. However, generating the code which is deeply hardware-specific, architecture-…
A Survey of Real-time Scheduling on Accelerator-based Heterogeneous Architecture for Time Critical Applications
An Zou, Yuankai Xu, Yinchen Ni +6
Accelerator-based heterogeneous architectures, such as CPU-GPU, CPU-TPU, and CPU-FPGA systems, are widely adopted to support the popular artificial intelligence (AI) algorithms tha…
Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
Jiace Zhu, Yuanzhe Huang, Yingtao Shen +2
To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, c…
Order of Compression: A Systematic and Optimal Sequence to Combinationally Compress CNN
Yingtao Shen, Minqing Sun, Jianzhe Lin +2
Model compression has gained significant popularity as a means to alleviate the computational and memory demands of machine learning models. Each compression technique leverages un…
ONE-SA: Enabling Nonlinear Operations in Systolic Arrays for Efficient and Flexible Neural Network Inference
Ruiqi Sun, Yinchen Ni, Xin He +2
The computation and memory-intensive nature of DNNs limits their use in many mobile and embedded contexts. Application-specific integrated circuit (ASIC) hardware accelerators empl…
NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inference
Ruiqi Sun, Siwei Ye, Jie Zhao +4
The inherent diversity of computation types within the deep neural network (DNN) models often requires a variety of specialized units in hardware processors, which limits computati…
Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
Xiangjie Li, Chenfei Lou, Zhengping Zhu +4
By adding exiting layers to the deep learning networks, early exit can terminate the inference earlier with accurate results. The passive decision-making of whether to exit or cont…
RTGPU: Real-Time GPU Scheduling of Hard Deadline Parallel Tasks with Fine-Grain Utilization
An Zou, Jing Li, Christopher D. Gill +1
Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system op…