7 papers
Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
Ziyi Chen, Peiran Yu, Heng Huang
This work aims to solve a stochastic nonconvex nonsmooth composite optimization problem. Previous works on composite optimization problem requires the major part to satisfy Lipschi…
Trade-off in Estimating the Number of Byzantine Clients in Federated Learning
Ziyi Chen, Su Zhang, Heng Huang
Federated learning has attracted increasing attention at recent large-scale optimization and machine learning research and applications, but is also vulnerable to Byzantine clients…
Achieve Performatively Optimal Policy for Performative Reinforcement Learning
Ziyi Chen, Heng Huang
Performative reinforcement learning is an emerging dynamical decision making framework, which extends reinforcement learning to the common applications where the agent's policy can…
Provably Mitigating Corruption, Overoptimization, and Verbosity Simultaneously in Offline and Online RLHF/DPO Alignment
Ziyi Chen, Junyi Li, Peiran Yu +1
Reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) are important techniques to align large language models (LLM) with human preference. Howe…
Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Duality
Shaocong Ma, Ziyi Chen, Yi Zhou +1
The goal of robust constrained reinforcement learning (RL) is to optimize an agent's performance under the worst-case model uncertainty while satisfying safety or resource constrai…
Revisiting Convergence: Shuffling Complexity Beyond Lipschitz Smoothness
Qi He, Peiran Yu, Ziyi Chen +1
Shuffling-type gradient methods are favored in practice for their simplicity and rapid empirical performance. Despite extensive development of convergence guarantees under various…