2 citations · 2 across the 5 of their papers we have counts for
9 papers
ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
Zhiwei Hao, Jianyuan Guo, Li Shen +4
Recent advancements in vision transformers (ViTs) have demonstrated that larger models often achieve superior performance. However, training these models remains computationally in…
LLM Data Selection and Utilization via Dynamic Bi-level Optimization
Yang Yu, Kai Han, Hang Zhou +4
While large-scale training data is fundamental for developing capable large language models (LLMs), strategically selecting high-quality data has emerged as a critical approach to…
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
Zhongqian Fu, Tianyi Zhao, Ning Ding +4
Mixture-of-Experts (MoE) models enable scalable computation and performance in large-scale deep learning but face quantization challenges due to sparse expert activation and dynami…
Transferable text data distillation by trajectory matching
Rong Yao, Hailin Hu, Yifei Fu +5
In the realm of large language model (LLM), as the size of large models increases, it also brings higher training costs. There is a urgent need to minimize the data size in LLM tra…
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
Ning Ding, Jing Han, Yuchuan Tian +3
Diffusion Transformer (DiT) has now become the preferred choice for building image generation models due to its great generation capability. Unlike previous convolution-based UNet…
SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
Shibo Jie, Yehui Tang, Kai Han +2
Transformer-based large language models (LLMs) have already achieved remarkable results on long-text tasks, but the limited GPU memory (VRAM) resources struggle to accommodate the…