36 citations · 77 across the 14 of their papers we have counts for
9 papers · 1 filter
SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
Han Shen, Pin-Yu Chen, Payel Das +1
Fine-tuning on task-specific data to boost downstream performance is a crucial step for leveraging Large Language Models (LLMs). However, previous studies have demonstrated that fi…
Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
Heshan Fernando, Han Shen, Parikshit Ram +4
The post-training of LLMs, which typically consists of the supervised fine-tuning (SFT) stage and the preference learning stage (RLHF or DPO), is crucial to effective and safe LLM…
Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
Han Shen, Zhuoran Yang, Tianyi Chen
Bilevel optimization has been recently applied to many machine learning tasks. However, their applications have been restricted to the supervised learning setting, where static obj…
Differentially Private Non-convex Learning for Multi-layer Neural Networks
Hanpu Shen, Cheng-Long Wang, Zihang Xiang +2
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node. In the first part,…
On Penalty-based Bilevel Gradient Descent Method
Han Shen, Quan Xiao, Tianyi Chen
Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, met…
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization
Quan Xiao, Han Shen, Wotao Yin +1
Stochastic bilevel optimization, which captures the inherent nested structure of machine learning problems, is gaining popularity in many recent applications. Existing works on bil…
Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
Heshan Fernando, Han Shen, Miao Liu +3
Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance met…
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences
Han Shen, Tianyi Chen
Stochastic approximation (SA) with multiple coupled sequences has found broad applications in machine learning such as bilevel learning and reinforcement learning (RL). In this pap…
Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup
Han Shen, Kaiqing Zhang, Mingyi Hong +1
Asynchronous and parallel implementation of standard reinforcement learning (RL) algorithms is a key enabler of the tremendous success of modern RL. Among many asynchronous RL algo…