3 citations · 6 across the 10 of their papers we have counts for
12 papers
Derivative-Free Structured Updates for Muon
Pengcheng Xie
Muon updates matrix-valued neural-network parameters by orthogonalizing a gradient-based momentum matrix. Its reliance on derivatives limits its use when gradients are unavailable…
MpSub: A Momentum -Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Yuyang Wang, Haoyu Yao, Pengcheng Xie
Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by…
TOBYQA: A Time-Augmented Model-Based Method for Derivative-Free Optimization under Noise and Temporal Drift
Haoyu Yao, Pengcheng Xie
Derivative-free optimization (DFO) is challenging when the observation channel varies over time and evaluations are noisy. Conventional model-based methods assume stationary observ…
Why and When Neural Networks Improve Local Approximation in Optimization
Chengkuo Bian, Pengcheng Xie
Published experience with neural surrogates in derivative-free optimisation is contradictory: the same family of models that cuts the evaluation count of one solver leaves another…
Parallel Model-Based Derivative-Free Optimization via Rank-Two KKT Updates
Donghan Wu, Pengcheng Xie
Derivative-free optimization (DFO) addresses unconstrained problems $\min_{\x\in\RR^n} f(\x)$ where is accessed only through a zeroth-order oracle. Model-based trust-region met…
Low-Rank KKT Updates and a Parallel Flipping Mechanism for Model-Based Derivative-Free Optimization
Donghan Wu, Pengcheng Xie
Model-based derivative-free optimization relies on quadratic interpolation, but maintaining these models typically requires linear system solves. We show that fo…
BUP-TR: Bayesian Underdetermined Projection Trust-Region Methods for Derivative-Free Optimization
Wei Hu, Pengcheng Xie, Ya-Xiang Yuan +1
Underdetermined quadratic interpolation is a central model-construction tool in model-based derivative-free trust-region methods: it limits sampling costs but leaves an affine fami…
MATRO: Metric-Aware Trust-Region Optimization with Fully Quadratic Models
Wei Hu, Pengcheng Xie, Ya-Xiang Yuan +1
Model-based derivative-free trust-region methods build local interpolation models and restrict trial steps to regions where those models are reliable. This paper studies the shape…
Distributed Gradient-Regularized Newton Method: Scheduled Consensus and O(epsilon^{-1}) Global Iteration Complexity
Wei Hu, Pengcheng Xie, Ya-Xiang Yuan +1
We propose DisGrem, a fully decentralized second-order method for convex consensus optimization over networks. Each agent solves a local Newton system with vanishing gradient-norm…
A New Two-dimensional Model-based Subspace Method for Large-scale Unconstrained Derivative-free Optimization: 2D-MoSub
Pengcheng Xie, Ya-xiang Yuan
This paper proposes the method 2D-MoSub (2-dimensional model-based subspace method), which is a novel derivative-free optimization (DFO) method based on the subspace method for gen…
Least Norm Updating Quadratic Interpolation Model Function for Derivative-free Trust-region Algorithms
Pengcheng Xie, Ya-xiang Yuan
Derivative-free optimization methods are numerical methods for optimization problems in which no derivative information is used. Such optimization problems are widely seen in many…
Derivative-Free Optimization with Transformed Objective Functions (DFOTO) and the Algorithm Based on the Least Frobenius Norm Updating Quadratic Model
Pengcheng Xie, Ya-xiang Yuan
Derivative-free optimization problems are optimization problems where derivative information is unavailable. The least Frobenius norm updating quadratic interpolation model functio…