186 citations · 756 across the 14 of their papers we have counts for
14 papers
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Sang Michael Xie, Hieu Pham, Xuanyi Dong +7
The mixture proportions of pretraining data domains (e.g., Wikipedia, books, web text) greatly affect language model (LM) performance. In this paper, we propose Domain Reweighting…
PyGlove: Efficiently Exchanging ML Ideas as Code
Daiyi Peng, Xuanyi Dong, Esteban Real +2
The increasing complexity and scale of machine learning (ML) has led to the need for more efficient collaboration among multiple teams. For example, when a research team invents a…
Symbolic Discovery of Optimization Algorithms
Xiangning Chen, Chen Liang, Da Huang +9
We present a method to formulate algorithm discovery as program search, and apply it to discover optimization algorithms for deep neural network training. We leverage efficient sea…
Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version
Razvan-Gabriel Cirstea, Chenjuan Guo, Bin Yang +3
A variety of real-world applications rely on far future information to make decisions, thus calling for efficient and accurate long sequence multivariate time series forecasting. W…
Automated Deep Learning: Neural Architecture Search Is Not the End
Xuanyi Dong, David Jacob Kedziora, Katarzyna Musial +1
Deep learning (DL) has proven to be a highly effective approach for developing models in diverse contexts, including visual perception, speech recognition, and machine translation.…
Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou, Xuanyi Dong, Berkin Akin +7
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed m…
Isometric Propagation Network for Generalized Zero-shot Learning
Lu Liu, Tianyi Zhou, Guodong Long +3
Zero-shot learning (ZSL) aims to classify images of an unseen class only based on a few attributes describing that class but no access to any training sample. A popular strategy is…
Supervision by Registration and Triangulation for Landmark Detection
Xuanyi Dong, Yi Yang, Shih-En Wei +3
We present Supervision by Registration and Triangulation (SRT), an unsupervised approach that utilizes unlabeled multi-view video to improve the accuracy and precision of landmark…
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
Xuanyi Dong, Lu Liu, Katarzyna Musial +1
Neural architecture search (NAS) has attracted a lot of attention and has been illustrated to bring tangible benefits in a large number of applications in the past few years. Archi…
AutoHAS: Efficient Hyperparameter and Architecture Search
Xuanyi Dong, Mingxing Tan, Adams Wei Yu +3
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architec…
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
Xuanyi Dong, Yi Yang
Neural architecture search (NAS) has achieved breakthrough success in a great number of applications in the past few years. It could be time to take a step back and analyze the goo…
Searching for A Robust Neural Architecture in Four GPU Hours
Xuanyi Dong, Yi Yang
Conventional neural architecture search (NAS) approaches are based on reinforcement learning or evolutionary strategy, which take more than 3000 GPU hours to find a good model on C…
Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks
Yang He, Xuanyi Dong, Guoliang Kang +3
Deeper and wider Convolutional Neural Networks (CNNs) achieve superior performance but bring expensive computation cost. Accelerating such over-parameterized neural network has rec…
Few-Example Object Detection with Model Communication
Xuanyi Dong, Liang Zheng, Fan Ma +2
In this paper, we study object detection using a large pool of unlabeled images and only a few labeled images per category, named "few-example object detection". The key challenge…