5 papers · 1 filter
Divergence based Ranked Preference Alignment for LLMs
Rajdeep Haldar, Yue Xing, Qifan Song
Alignment of large language models typically relies on pairwise preference feedback, though ranked feedback can capture more comprehensive information across multiple responses wit…
CaliCausalRank: Calibrated Multi-Objective Ad Ranking with Robust Counterfactual Utility Optimization
Xikai Yang, Sebastian Sun, Yilin Li +3
Ad ranking systems must simultaneously optimize multiple objectives including click-through rate (CTR), conversion rate (CVR), revenue, and user experience metrics. However, produc…
f-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment
Rajdeep Haldar, Lantao Mei, Guang Lin +2
Recent work shows that preference alignment objectives can be interpreted as divergence estimators between aligned (preferred) & unaligned (less-preferred) distributions, yielding…
Contrastive Learning-Based Dependency Modeling for Anomaly Detection in Cloud Services
Yue Xing, Yingnan Deng, Heyao Liu +3
This paper addresses the challenges of complex dependencies and diverse anomaly patterns in cloud service environments by proposing a dependency modeling and anomaly detection meth…
How to Enhance Downstream Adversarial Robustness (almost) without Touching the Pre-Trained Foundation Model?
Meiqi Liu, Zhuoqun Huang, Yue Xing
With the rise of powerful foundation models, a pre-training-fine-tuning paradigm becomes increasingly popular these days: A foundation model is pre-trained using a huge amount of d…