6 papers
River-LLM: Large Language Model Seamless Exit Based on KV Share
Yingtao Shen, An Zou
Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency. Early Exit has emerged as…
CUDABench: Benchmarking LLMs for Text-to-CUDA Generation
Jiace Zhu, Wentao Chen, Qi Fan +6
Recent studies have demonstrated the potential of Large Language Models (LLMs) in generating GPU Kernels. Current benchmarks focus on the translation of high-level languages into C…
TimeBill: Time-Budgeted Inference for Large Language Models
Qi Fan, An Zou, Yehan Ma
Large Language Models (LLMs) are increasingly deployed in time-critical systems, such as robotics, autonomous driving, embodied intelligence, and industrial automation, where gener…
Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
Jiace Zhu, Yuanzhe Huang, Yingtao Shen +2
To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, c…
CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
Wentao Chen, Jiace Zhu, Qi Fan +2
Large Language Models (LLMs) have demonstrated strong capabilities in general-purpose code generation. However, generating the code which is deeply hardware-specific, architecture-…
A Survey of Real-time Scheduling on Accelerator-based Heterogeneous Architecture for Time Critical Applications
An Zou, Yuankai Xu, Yinchen Ni +6
Accelerator-based heterogeneous architectures, such as CPU-GPU, CPU-TPU, and CPU-FPGA systems, are widely adopted to support the popular artificial intelligence (AI) algorithms tha…