Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
HiFloat4 Format for Language Model Inference
Yuanyong Luo, Jing Huang, Yu Cheng +19
This paper introduces HiFloat4 (HiF4), a block floating-point data format tailored for deep learning. Each HiF4 unit packs 64 4-bit elements with 32 bits of shared scaling metadata…
cs.LG2025
Transfer Learning on Edge Connecting Probability Estimation under Graphon Model
Yuyao Wang, Yu-Hung Cheng, Debarghya Mukherjee +1
Graphon models provide a flexible nonparametric framework for estimating latent connectivity probabilities in networks, enabling a range of downstream applications such as link pre…
cs.LG2024
MoE-I: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
Cheng Yang, Yang Sui, Jinqi Xiao +7
The emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models. Compared to traditional LLMs, MoE LLMs outperform traditional LLMs by…