4 papers
Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
Jiace Zhu, Yuanzhe Huang, Yingtao Shen +2
To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, c…
Order of Compression: A Systematic and Optimal Sequence to Combinationally Compress CNN
Yingtao Shen, Minqing Sun, Jianzhe Lin +2
Model compression has gained significant popularity as a means to alleviate the computational and memory demands of machine learning models. Each compression technique leverages un…
ONE-SA: Enabling Nonlinear Operations in Systolic Arrays for Efficient and Flexible Neural Network Inference
Ruiqi Sun, Yinchen Ni, Xin He +2
The computation and memory-intensive nature of DNNs limits their use in many mobile and embedded contexts. Application-specific integrated circuit (ASIC) hardware accelerators empl…
NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inference
Ruiqi Sun, Siwei Ye, Jie Zhao +4
The inherent diversity of computation types within the deep neural network (DNN) models often requires a variety of specialized units in hardware processors, which limits computati…