10 papers
ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
Xiaoqiang Lin, Arun Verma, Zhongxiang Dai +3
The recent success in using human preferences to align large language models (LLMs) has significantly improved their performance in various downstream tasks, such as question answe…
Active Human Feedback Collection via Neural Contextual Dueling Bandits
Arun Verma, Xiaoqiang Lin, Zhongxiang Dai +2
Collecting human preference feedback is often expensive, leading recent works to develop principled algorithms to select them more efficiently. However, these works assume that the…
Refining Adaptive Zeroth-Order Optimization at Ease
Yao Shu, Qixin Zhang, Kun He +1
Recently, zeroth-order (ZO) optimization plays an essential role in scenarios where gradient information is inaccessible or unaffordable, such as black-box systems and resource-con…
Neural Dueling Bandits: Preference-Based Optimization with Human Feedback
Arun Verma, Zhongxiang Dai, Xiaoqiang Lin +2
Contextual dueling bandit is used to model the bandit problems, where a learner's goal is to find the best arm for a given context using observed noisy human preference feedback ov…
Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars
Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai +5
Large language models (LLMs) have shown impressive capabilities in real-world applications. The capability of in-context learning (ICL) allows us to adapt an LLM to downstream task…
Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimization
Shouri Hu, Haowei Wang, Zhongxiang Dai +2
The expected improvement (EI) is one of the most popular acquisition functions for Bayesian optimization (BO) and has demonstrated good empirical performances in many applications…