19 citations · 57 across the 18 of their papers we have counts for
18 papers
Refining Adaptive Zeroth-Order Optimization at Ease
Yao Shu, Qixin Zhang, Kun He +1
Recently, zeroth-order (ZO) optimization plays an essential role in scenarios where gradient information is inaccessible or unaffordable, such as black-box systems and resource-con…
Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models
Yao Shu, Wenyang Hu, See-Kiong Ng +2
Large Language Models (LLMs) have become indispensable in numerous real-world applications. However, fine-tuning these models at scale, especially in federated settings where data…
Flexora: Flexible Low Rank Adaptation for Large Language Models
Chenxing Wei, Yao Shu, Ying Tiffany He +1
Large Language Models (LLMs) are driving advancements in artificial intelligence by increasing the scale of model parameters, which has significantly enhanced generalization abilit…
Localized Zeroth-Order Prompt Optimization
Wenyang Hu, Yao Shu, Zongmin Yu +5
The efficacy of large language models (LLMs) in understanding and generating natural language has aroused a wide interest in developing prompt-based methods to harness the power of…
OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations
Yao Shu, Jiongfeng Fang, Ying Tiffany He +1
First-order optimization (FOO) algorithms are pivotal in numerous computational domains such as machine learning and signal denoising. However, their application to complex tasks l…
Exploiting Correlated Auxiliary Feedback in Parameterized Bandits
Arun Verma, Zhongxiang Dai, Yao Shu +1
We study a novel variant of the parameterized bandits problem in which the learner can observe additional auxiliary feedback that is correlated with the observed reward. The auxili…
Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers
Xiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai +5
Large language models (LLMs) have shown remarkable instruction-following capabilities and achieved impressive performances in various applications. However, the performances of LLM…
Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients
Yao Shu, Xiaoqiang Lin, Zhongxiang Dai +1
Federated optimization, an emerging paradigm which finds wide real-world applications such as federated learning, enables multiple clients (e.g., edge devices) to collaboratively o…
Sample-Then-Optimize Batch Neural Thompson Sampling
Zhongxiang Dai, Yao Shu, Bryan Kian Hsiang Low +1
Bayesian optimization (BO), which uses a Gaussian process (GP) as a surrogate to model its objective function, is popular for black-box optimization. However, due to the limitation…
Federated Neural Bandits
Zhongxiang Dai, Yao Shu, Arun Verma +3
Recent works on neural contextual bandits have achieved compelling performances due to their ability to leverage the strong representation power of neural networks (NNs) for reward…
Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search
Yao Shu, Zhongxiang Dai, Zhaoxuan Wu +1
Neural architecture search (NAS) has gained immense popularity owing to its ability to automate neural architecture design. A number of training-free metrics are recently proposed…
Neural Ensemble Search via Bayesian Sampling
Yao Shu, Yizhou Chen, Zhongxiang Dai +1
Recently, neural architecture search (NAS) has been applied to automate the design of neural networks in real-world applications. A large number of algorithms have been developed t…
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization
Yao Shu, Shaofeng Cai, Zhongxiang Dai +2
Recent years have witnessed a surging interest in Neural Architecture Search (NAS). Various algorithms have been proposed to improve the search efficiency and effectiveness of NAS,…
Tight Lower Complexity Bounds for Strongly Convex Finite-Sum Optimization
Min Zhang, Yao Shu, Kun He
Finite-sum optimization plays an important role in the area of machine learning, and hence has triggered a surge of interest in recent years. To address this optimization problem,…
Understanding Architectures Learnt by Cell-based Neural Architecture Search
Yao Shu, Wei Wang, Shaofeng Cai
Neural architecture search (NAS) searches architectures automatically for given tasks, e.g., image classification and language modeling. Improving the search efficiency and effecti…
Dynamic Routing Networks
Shaofeng Cai, Yao Shu, Wei Wang +1
The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs. Extensive efforts have been made to improve the accuracy with…
Efficient Memory Management for GPU-based Deep Learning Systems
Junzhe Zhang, Sai Ho Yeung, Yao Shu +2
GPU (graphics processing unit) has been used for many data-intensive applications. Among them, deep learning systems are one of the most important consumer systems for GPU nowadays…
Randomness in Deconvolutional Networks for Visual Representation
Kun He, Jingbo Wang, Haochuan Li +5
Toward a deeper understanding on the inner work of deep neural networks, we investigate CNN (convolutional neural network) using DCN (deconvolutional network) and randomization tec…