10 papers
Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Licheng Pan, Haocheng Yang, Haoxuan Li +7
Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM v…
Not All Criteria Matter: Importance-Aware Criteria Weighting for Rubric-Based LLM Evaluation
Haocheng Yang, Licheng Pan, Yiming Dong +7
Rubric-based evaluation assesses the responses of large language models (LLMs) by aggregating the verification results of explicit criteria, yet not all criteria matter equally. Ho…
Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence
Haocheng Yang, Licheng Pan, Xiaoxi Li +8
Rubric evolution offers a promising approach to improving the quality of rubrics generated by large language models (LLMs). Central to this process is rubric comparison, which iden…
MuSEAgent: A Multimodal Reasoning Agent with Stateful Experiences
Shijian Wang, Jiarui Jin, Runhao Fu +11
Research agents have recently achieved significant progress in information seeking and synthesis across heterogeneous textual and visual sources. In this paper, we introduce MuSEAg…
Unbiased Reward Modeling from Implicit Feedback for LLM Alignment
Hao Wang, Haocheng Yang, Licheng Pan +7
Despite the success of reinforcement learning from human feedback (RLHF), existing reward modeling methods largely rely on explicit feedback, which is costly to collect and difficu…
Deep Time-Series Forecasting in 10 Years: A Survey
Hao Wang, Licheng Pan, Qingsong Wen +12
Autocorrelation is a common property of time-series, where each observation is dependent on its predecessors. In deep time-series forecasting, it raises two central challenges: (1)…
CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
Hao Wang, Licheng Pan, Zhichao Chen +7
Despite the success of reinforcement learning from human feedback (RLHF) in aligning language models, current reward modeling heavily relies on experimental feedback data collected…
Deep Time-series Forecasting Needs Kernelized Moment Balancing
Licheng Pan, Hao Wang, Haocheng Yang +12
Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths. According to Imbens' crite…
TourPlanner: A Competitive Consensus Framework with Constraint-Gated Reinforcement Learning for Travel Planning
Yinuo Wang, Mining Tan, Wenxiang Jiao +5
Travel planning is a sophisticated decision-making process that requires synthesizing multifaceted information to construct itineraries. However, existing travel planning approache…
DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignment
Hao Wang, Licheng Pan, Yuan Lu +7
Training time-series forecasting models requires aligning the conditional distribution of model forecasts with that of the label sequence. The standard direct forecast (DF) approac…