133 citations · 194 across the 7 of their papers we have counts for
4 papers · 1 filter
BCN: Batch Channel Normalization for Image Classification
Afifa Khaled, Chao Li, Jia Ning +1
Normalization techniques have been widely used in the field of deep learning due to their capability of enabling higher learning rates and are less careful in initialization. Howev…
FP8-LM: Training FP8 Large Language Models
Houwen Peng, Kan Wu, Yixuan Wei +17
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer s…
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
Jia Ning, Chen Li, Zheng Zhang +4
Unlike language tasks, where the output space is usually limited to a set of tokens, the output space of visual tasks is more complicated, making it difficult to build a unified vi…
GMConv: Modulating Effective Receptive Fields for Convolutional Kernels
Qi Chen, Chao Li, Jia Ning +2
In convolutional neural networks, the convolutions are conventionally performed using a square kernel with a fixed N N receptive field (RF). However, what matters most to…