99 citations · 141 across the 18 of their papers we have counts for
3 papers · 1 filter
On the Spectral Flattening of Quantized Embeddings
Junlin Huang, Wenyi Fang, Zhenheng Tang +5
Training Large Language Models (LLMs) at ultra-low precision is critically impeded by instability rooted in the conflict between discrete quantization constraints and the intrinsic…
FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
Zhenheng Tang, Xiaowen Chu, Ryan Yide Ran +7
Federated Learning (FL) enables collaborations among clients for train machine learning models while protecting their data privacy. Existing FL simulation platforms that are design…
A Survey of Transformers
Tianyang Lin, Yuxin Wang, Xiangyang Liu +1
Transformers have achieved great success in many artificial intelligence fields, such as natural language processing, computer vision, and audio processing. Therefore, it is natura…