13 citations · 30 across the 15 of their papers we have counts for
15 papers
Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback
Yikai Wang, Shang Liu, Jose Blanchet
Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human…
Incentivizing High-Quality Human Annotations with Golden Questions
Shang Liu, Zhongze Cai, Hanzhao Wang +2
Human-annotated data plays a vital role in training large language models (LLMs), such as supervised fine-tuning and human preference alignment. However, it is not guaranteed that…
How Humans Help LLMs: Assessing and Incentivizing Human Preference Annotators
Shang Liu, Hanzhao Wang, Zhongyao Ma +1
Human-annotated preference data play an important role in aligning large language models (LLMs). In this paper, we study two connected questions: how to monitor the quality of huma…
Reward Modeling with Ordinal Feedback: Wisdom of the Crowd
Shang Liu, Yu Pan, Guanting Chen +1
Learning a reward model (RM) from human preferences has been an important component in aligning large language models (LLMs). The canonical setup of learning RMs from pairwise pref…
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification
Shang Liu, Zhongze Cai, Guanting Chen +1
Predicting simple function classes has been widely used as a testbed for developing theory and understanding of the trained Transformer's in-context learning (ICL) ability. In this…
Understanding the Training and Generalization of Pretrained Transformer for Sequential Decision Making
Hanzhao Wang, Yu Pan, Fupeng Sun +4
In this paper, we consider the supervised pre-trained transformer for a class of sequential decision-making problems. The class of considered problems is a subset of the general fo…
Towards Better Statistical Understanding of Watermarking LLMs
Zhongze Cai, Shang Liu, Hanzhao Wang +2
In this paper, we study the problem of watermarking large language models (LLMs). We consider the trade-off between model distortion and detection ability and formulate it as a con…
RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model
Yao Lu, Shang Liu, Qijun Zhang +1
Inspired by the recent success of large language models (LLMs) like ChatGPT, researchers start to explore the adoption of LLMs for agile hardware design, such as generating design…
When No-Rejection Learning is Consistent for Regression with Rejection
Xiaocheng Li, Shang Liu, Chunlin Sun +1
Learning with rejection has been a prototypical model for studying the human-AI interaction on prediction tasks. Upon the arrival of a sample instance, the model first uses a rejec…
Understanding Uncertainty Sampling via Equivalent Loss
Shang Liu, Xiaocheng Li
Uncertainty sampling is a classical active-learning strategy, yet the statistical objective induced by its query rule is often implicit. We introduce the equivalent loss, whose gra…
Distribution-Free Model-Agnostic Regression Calibration via Nonparametric Methods
Shang Liu, Zhongze Cai, Xiaocheng Li
In this paper, we consider the uncertainty quantification problem for regression models. Specifically, we consider an individual calibration objective for characterizing the quanti…
Maximum Optimality Margin: A Unified Approach for Contextual Linear Programming and Inverse Linear Programming
Chunlin Sun, Shang Liu, Xiaocheng Li
In this paper, we study the predict-then-optimize problem where the output of a machine learning prediction task is used as the input of some downstream optimization problem, say,…
Trixi the Librarian
Fabian Wieczorek, Shang-Ching Liu, Björn Sygo +1
In this work, we present a three-part system that automatically sorts books on a shelf using the PR- 2 platform. The paper describes a methodology to sufficiently detect and recogn…
Non-stationary Bandits with Knapsacks
Shang Liu, Jiashuo Jiang, Xiaocheng Li
In this paper, we study the problem of bandits with knapsacks (BwK) in a non-stationary environment. The BwK problem generalizes the multi-arm bandit (MAB) problem to model the res…
Online Bin Packing with Known T
Shang Liu, Xiaocheng Li
In the online bin packing problem, a sequence of items is revealed one at a time, and each item must be packed into an available bin instantly upon its arrival. In this paper, we r…