6 citations · 17 across the 9 of their papers we have counts for
9 papers
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization
Weibo Zhao, Yubin Shi, Xinyu Lyu +3
Quantization stands as a pivotal technique for large language model (LLM) serving, yet it poses significant challenges particularly in achieving effective low-bit quantization. The…
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Yunshan Zhong, Yuyao Zhou, Yuxin Zhang +5
Data-free quantization (DFQ) enables model quantization without accessing real data, addressing concerns regarding data security and privacy. With the growing adoption of Vision Tr…
DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization
Yanfeng Jiang, Zelan Yang, Bohua Chen +3
Large language models achieve exceptional performance on various downstream tasks through supervised fine-tuning. However, the diversity of downstream tasks and practical requireme…
Safety Layers in Aligned Large Language Models: The Key to LLM Security
Shen Li, Liuyi Yao, Lan Zhang +1
Aligned LLMs are secure, capable of recognizing and refusing to answer malicious questions. However, the role of internal parameters in maintaining such security is not well unders…
Effectively Compress KV Heads for LLM
Hao Yu, Zelan Yang, Shen Li +2
The advent of pre-trained large language models (LLMs) has revolutionized various natural language processing tasks. These models predominantly employ an auto-regressive decoding m…
Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
Shen Li, Liuyi Yao, Jinyang Gao +2
To support various applications, a prevalent and efficient approach for business owners is leveraging their valuable datasets to fine-tune a pre-trained LLM through the API provide…
Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
Bin Lin, Chen Zhang, Tao Peng +12
Large Language Models (LLMs) demonstrate substantial potential across a diverse array of domains via request serving. However, as trends continue to push for expanding context size…
1xN Pattern for Pruning Convolutional Neural Networks
Mingbao Lin, Yuxin Zhang, Yuchao Li +6
Though network pruning receives popularity in reducing the complexity of convolutional neural networks (CNNs), it remains an open issue to concurrently maintain model accuracy as w…