84 citations · 94 across the 12 of their papers we have counts for
12 papers
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling
Jiahao Qiu, Yifu Lu, Yifan Zeng +9
Inference-time alignment enhances the performance of large language models without requiring additional training or fine-tuning but presents challenges due to balancing computation…
A Common Pitfall of Margin-based Language Model Alignment: Gradient Entanglement
Hui Yuan, Yifan Zeng, Yue Wu +3
Reinforcement Learning from Human Feedback (RLHF) has become the predominant approach for language model (LM) alignment. At its core, RLHF uses a margin-based loss for preference o…
Conversational Dueling Bandits in Generalized Linear Models
Shuhua Yang, Hui Yuan, Xiaoying Zhang +3
Conversational recommendation systems elicit user preferences by interacting with users to obtain their feedback on recommended commodities. Such systems utilize a multi-armed band…
Contractual Reinforcement Learning: Pulling Arms with Invisible Hands
Jibang Wu, Siyu Chen, Mengdi Wang +2
The agency problem emerges in today's large scale machine learning tasks, where the learners are unable to direct content creation or enforce data collection. In this work, we prop…
Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization
Jiahao Qiu, Hui Yuan, Jinghong Zhang +3
While modern biotechnologies allow synthesizing new proteins and function measurements at scale, efficiently exploring a protein sequence space and engineering it remains a dauntin…
PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback
Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel +4
We present a novel unified bilevel optimization-based framework, \textsf{PARL}, formulated to address the recently highlighted critical issue of policy alignment in reinforcement l…
How Does Diffusion Influence Pretrained Language Models on Out-of-Distribution Data?
Huazheng Wang, Daixuan Cheng, Haifeng Sun +5
Transformer-based pretrained language models (PLMs) have achieved great success in modern NLP. An important advantage of PLMs is good out-of-distribution (OOD) robustness. Recently…
Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems
Xiang Ji, Huazheng Wang, Minshuo Chen +2
For a real-world decision-making problem, the reward function often needs to be engineered or learned. A popular approach is to utilize human feedback to learn a reward function fo…
Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP
Jiacheng Guo, Zihao Li, Huazheng Wang +3
In this paper, we study representation learning in partially observable Markov Decision Processes (POMDPs), where the agent learns a decoder function that maps a series of high-dim…
Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
Zeyu Zhang, Yi Su, Hui Yuan +5
Off-policy Learning to Rank (LTR) aims to optimize a ranker from data collected by a deployed logging policy. However, existing off-policy learning to rank methods often make stron…
Adversarial Attacks on Online Learning to Rank with Stochastic Click Models
Zichen Wang, Rishab Balasubramanian, Hui Yuan +3
We propose the first study of adversarial attacks on online learning to rank. The goal of the adversary is to misguide the online learning to rank algorithm to place the target ite…
Machine Learning for Synthetic Data Generation: A Review
Yingzhou Lu, Lulu Chen, Yuanyuan Zhang +6
Machine learning heavily relies on data, but real-world applications often encounter various data-related issues. These include data of poor quality, insufficient data points leadi…