4 papers
Optimizing Large Language Model Training Using FP4 Quantization
Ruizhe Wang, Yeyun Gong, Xiao Liu +5
The growing computational demands of training large language models (LLMs) necessitate more efficient methods. Quantized training presents a promising solution by enabling low-bit…
SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation
Yifan Xiong, Yuting Jiang, Ziyue Yang +17
Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently…
Phoneme-Based Proactive Anti-Eavesdropping with Controlled Recording Privilege
Peng Huang, Yao Wei, Peng Cheng +5
The widespread smart devices raise people's concerns of being eavesdropped on. To enhance voice privacy, recent studies exploit the nonlinearity in microphone to jam audio recorder…
FP8-LM: Training FP8 Large Language Models
Houwen Peng, Kan Wu, Yixuan Wei +17
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer s…