133 citations · 194 across the 7 of their papers we have counts for
7 papers
BCN: Batch Channel Normalization for Image Classification
Afifa Khaled, Chao Li, Jia Ning +1
Normalization techniques have been widely used in the field of deep learning due to their capability of enabling higher learning rates and are less careful in initialization. Howev…
FP8-LM: Training FP8 Large Language Models
Houwen Peng, Kan Wu, Yixuan Wei +17
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer s…
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
Jia Ning, Chen Li, Zheng Zhang +4
Unlike language tasks, where the output space is usually limited to a set of tokens, the output space of visual tasks is more complicated, making it difficult to build a unified vi…
GMConv: Modulating Effective Receptive Fields for Convolutional Kernels
Qi Chen, Chao Li, Jia Ning +2
In convolutional neural networks, the convolutions are conventionally performed using a square kernel with a fixed N N receptive field (RF). However, what matters most to…
Adaptive Channel Allocation for Robust Differentiable Architecture Search
Chao Li, Jia Ning, Han Hu +1
Differentiable ARchiTecture Search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency. However, the excessive accumulation of the…
Swin Transformer V2: Scaling Up Capacity and Resolution
Ze Liu, Han Hu, Yutong Lin +9
Large-scale NLP models have been shown to significantly improve the performance on language tasks with no signs of saturation. They also demonstrate amazing few-shot capabilities l…
Video Swin Transformer
Ze Liu, Jia Ning, Yue Cao +4
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchm…