71 citations · 81 across the 15 of their papers we have counts for
9 papers · 1 filter
Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
Xinxin Liu, Aaron Thomas, Cheng Zhang +3
Parameter-Efficient Fine-Tuning (PEFT) has gained prominence through low-rank adaptation methods like LoRA. In this paper, we focus on sparsity-based PEFT (SPEFT), which introduces…
Hardware and Software Platform Inference
Cheng Zhang, Hanna Foerster, Robert D. Mullins +2
It is now a common business practice to buy access to large language model (LLM) inference rather than self-host, because of significant upfront hardware infrastructure and energy…
Scaling Laws For Mixed Quantization
Zeyu Cao, Boyang Gu, Cheng Zhang +5
Post-training quantization of Large Language Models (LLMs) has proven effective in reducing the memory and computational requirements for inference. In this study, we focus on a st…
QERA: an Analytical Framework for Quantization Error Reconstruction
Cheng Zhang, Jeffrey T. H. Wong, Can Xiao +2
The growing number of parameters and computational demands of large language models (LLMs) present significant challenges for their efficient deployment. Recently, there is an incr…
Optimised Grouped-Query Attention Mechanism for Transformers
Yuang Chen, Cheng Zhang, Xitong Gao +3
Grouped-query attention (GQA) has been widely adopted in LLMs to mitigate the complexity of multi-head attention (MHA). To transform an MHA to a GQA, neighbour queries in MHA are e…
Unlocking the Global Synergies in Low-Rank Adapters
Zixi Zhang, Cheng Zhang, Xitong Gao +3
Low-rank Adaption (LoRA) has been the de-facto parameter-efficient fine-tuning technique for large language models. We present HeteroLoRA, a light-weight search algorithm that leve…
HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator
Zhewen Yu, Sudarshan Sreeram, Krish Agrawal +6
Deep Neural Networks (DNNs) excel in learning hierarchical representations from raw data, such as images, audio, and text. To compute these DNN models with high performance and ene…
Locking Machine Learning Models into Hardware
Eleanor Clifford, Adhithya Saravanan, Harry Langford +5
Modern machine learning (ML) models are expensive IP and business competitiveness often depends on keeping this IP confidential. This in turn restricts how these models are deploye…
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Cheng Zhang, Jianyi Cheng, George A. Constantinides +1
Post-training quantization of Large Language Models (LLMs) is challenging. In this work, we introduce Low-rank Quantization Error Reduction (LQER), which combines quantization and…