4 citations · 11 across the 16 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023★ 3 cited
FP8-LM: Training FP8 Large Language Models
Houwen Peng, Kan Wu, Yixuan Wei +17
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer s…
cs.CV2023★ 4 cited
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
Kan Wu, Houwen Peng, Zhenghong Zhou +10
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: af…