9 papers
Calibrating conditional risk
Andrey Vasilyev, Yikai Wang, Xiaocheng Li +1
We introduce and study the problem of calibrating conditional risk, which involves estimating the expected loss of a prediction model conditional on input features. We analyze this…
Risk Profiling and Modulation for LLMs
Yikai Wang, Xiaocheng Li, Guanting Chen
Large language models (LLMs) are increasingly used for decision-making tasks under uncertainty; however, their risk profiles and how they are influenced by prompting and alignment…
Collaborative Prediction: To Join or To Disjoin Datasets
Kyung Rok Kim, Yansong Wang, Xiaocheng Li +1
With the recent rise of generative Artificial Intelligence (AI), the need of selecting high-quality dataset to improve machine learning models has garnered increasing attention. Ho…
OMGPT: A Sequence Modeling Framework for Data-driven Operational Decision Making
Hanzhao Wang, Guanting Chen, Kalyan Talluri +1
We build a Generative Pre-trained Transformer (GPT) model from scratch to solve sequential decision making tasks arising in contexts of operations research and management science w…
Reward Modeling with Ordinal Feedback: Wisdom of the Crowd
Shang Liu, Yu Pan, Guanting Chen +1
Learning a reward model (RM) from human preferences has been an important component in aligning large language models (LLMs). The canonical setup of learning RMs from pairwise pref…
Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach
Linyu Liu, Yu Pan, Xiaocheng Li +1
In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored are…