605 citations · 919 across the 15 of their papers we have counts for
15 papers
An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation
Ziquan Liu, Yi Xu, Yuanhong Xu +5
The performance of machine learning models under distribution shift has been the focus of the community in recent years. Most of current methods have been proposed to improve the r…
Improved Fine-Tuning by Better Leveraging Pre-Training Data
Ziquan Liu, Yi Xu, Yuanhong Xu +5
As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in many deep learning applications, especially for small data sets. However, recent studie…
A linearized framework and a new benchmark for model selection for fine-tuning
Aditya Deshpande, Alessandro Achille, Avinash Ravichandran +6
Fine-tuning from a collection of models pre-trained on different domains (a "model zoo") is emerging as a technique to improve test accuracy in the low-data regime. However, model…
WeMix: How to Better Utilize Data Augmentation
Yi Xu, Asaf Noy, Ming Lin +3
Data augmentation is a widely used training trick in deep learning to improve the network generalization ability. Despite many encouraging results, several recent studies did point…
Learning Accurate Entropy Model with Global Reference for Image Compression
Yichen Qian, Zhiyu Tan, Xiuyu Sun +5
In recent deep image compression neural networks, the entropy model plays a critical role in estimating the prior distribution of deep image encodings. Existing methods combine hyp…
Semi-Anchored Detector for One-Stage Object Detection
Lei Chen, Qi Qian, Hao Li
A standard one-stage detector is comprised of two tasks: classification and regression. Anchors of different shapes are introduced for each location in the feature map to mitigate…
Improved Knowledge Distillation via Full Kernel Matrix Transfer
Qi Qian, Hao Li, Juhua Hu
Knowledge distillation is an effective way for model compression in deep learning. Given a large model (i.e., teacher model), it aims to improve the performance of a compact model…
Neural Architecture Design for GPU-Efficient Networks
Ming Lin, Hesen Chen, Xiuyu Sun +3
Many mission-critical systems are based on GPU for inference. It requires not only high recognition accuracy but also low latency in responding time. Although many studies are devo…
Towards Understanding Label Smoothing
Yi Xu, Yuanhong Xu, Qi Qian +2
Label smoothing regularization (LSR) has a great success in training deep neural networks by stochastic algorithms such as stochastic gradient descent and its variants. However, th…
Rethinking the Hyperparameters for Fine-tuning
Hao Li, Pratik Chaudhari, Hao Yang +4
Fine-tuning from pre-trained ImageNet models has become the de-facto standard for various computer vision tasks. Current practices for fine-tuning typically involve selecting an ad…
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling
Qi Qian, Lei Shang, Baigui Sun +3
Distance metric learning (DML) is to learn the embeddings where examples from the same class are closer than examples from different classes. It can be cast as an optimization prob…
MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models
Boyuan Pan, Yazheng Yang, Hao Li +4
Machine Comprehension (MC) is one of the core problems in natural language processing, requiring both understanding of the natural language and knowledge about the world. Rapid pro…
DR Loss: Improving Object Detection by Distributional Ranking
Qi Qian, Lei Chen, Hao Li +1
Most of object detection algorithms can be categorized into two classes: two-stage detectors and one-stage detectors. Recently, many efforts have been devoted to one-stage detector…
Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement
Ming Lin, Xiaomin Song, Qi Qian +4
Satellite-based positioning system such as GPS often suffers from large amount of noise that degrades the positioning accuracy dramatically especially in real-time applications. In…
Visualizing the Loss Landscape of Neural Nets
Hao Li, Zheng Xu, Gavin Taylor +2
Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions. It is well-known that certain network architecture designs (e.g., skip…