168 citations · 331 across the 12 of their papers we have counts for
12 papers
Why Does Sharpness-Aware Minimization Generalize Better Than SGD?
Zixiang Chen, Junkai Zhang, Yiwen Kou +3
The challenge of overfitting, in which the model memorizes the training data and fails to generalize to test data, has become increasingly significant in the training of large neur…
Red Teaming Language Model Detectors with Language Models
Zhouxing Shi, Yihan Wang, Fan Yin +3
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially decept…
Symbol tuning improves in-context learning in language models
Jerry Wei, Le Hou, Andrew Lampinen +8
We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrar…
Symbolic Discovery of Optimization Algorithms
Xiangning Chen, Chen Liang, Da Huang +9
We present a method to formulate algorithm discovery as program search, and apply it to discover optimization algorithms for deep neural network training. We leverage efficient sea…
Towards Efficient and Scalable Sharpness-Aware Minimization
Yong Liu, Siqi Mai, Xiangning Chen +2
Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training larg…
Can Vision Transformers Perform Convolution?
Shanda Li, Xiangning Chen, Di He +1
Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vi…
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving
Ruochen Wang, Xiangning Chen, Minhao Cheng +2
Predictor-based algorithms have achieved remarkable performance in the Neural Architecture Search (NAS) tasks. However, these methods suffer from high computation costs, as trainin…
Rethinking Architecture Selection in Differentiable NAS
Ruochen Wang, Minhao Cheng, Xiangning Chen +2
Differentiable Neural Architecture Search is one of the most popular Neural Architecture Search (NAS) methods for its search efficiency and simplicity, accomplished by jointly opti…
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations
Xiangning Chen, Cho-Jui Hsieh, Boqing Gong
Vision Transformers (ViTs) and MLPs signal further efforts on replacing hand-wired features or inductive biases with general-purpose neural architectures. Existing works empower th…
Concurrent Adversarial Learning for Large-Batch Training
Yong Liu, Xiangning Chen, Minhao Cheng +2
Large-batch training has become a commonly used technique when training neural networks with a large number of GPU/TPU processors. As batch size increases, stochastic optimizers te…
2.5D Visual Relationship Detection
Yu-Chuan Su, Soravit Changpinyo, Xiangning Chen +8
Visual 2.5D perception involves understanding the semantics and geometry of a scene through reasoning about object relationships with respect to the viewer in an environment. Howev…
Robust and Accurate Object Detection via Adversarial Learning
Xiangning Chen, Cihang Xie, Mingxing Tan +3
Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection. Noting that most s…