4 papers · 1 filter
CUDA-Harness: Harnessing Agentic CUDA Kernel Generation and Optimization from Natural Language
Qi Fan, An Zou, Yehan Ma
Developing high-performance CUDA kernels demands specialized knowledge in algorithm implementation, correctness validation, and hardware-aware parallel optimization, creating a sub…
River-LLM: Large Language Model Seamless Exit Based on KV Share
Yingtao Shen, An Zou
Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency. Early Exit has emerged as…
TimeBill: Time-Budgeted Inference for Large Language Models
Qi Fan, An Zou, Yehan Ma
Large Language Models (LLMs) are increasingly deployed in time-critical systems, such as robotics, autonomous driving, embodied intelligence, and industrial automation, where gener…
Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
Jiace Zhu, Yuanzhe Huang, Yingtao Shen +2
To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, c…