7 papers
Composite and Stochastic Smoothed Decentralized Bregman Proximal-Gradient Methods
Shengze Xu, Chenhan Jin, Qingsong Wang +1
Smoothed decentralized Bregman proximal-gradient (SDBPG) methods combine relative-smoothness geometry with edge-dual consensus. We extend SDBPG to node-specific convex nonsmooth re…
More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses
Ziyang Xu, Haitian Zhong, Hao Zhou +5
Automated generation of LLM harnesses promises to improve inference through task specialization. Yet additional answer coverage can arise from repeated execution of the same progra…
Adaptive Bregman Proximal Stochastic Gradient with a Stabilized Barzilai--Borwein Step Size
Chenhan Jin, Shengze Xu, Binghui Xie +4
Bregman proximal stochastic gradient (BPSG) methods bring variance-reduced composite optimization to objectives whose geometry is poorly captured by Euclidean smoothness. Their per…
Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
Junkai Lin, Junkai Chen, Siqi Hou +6
Multimodal large language models (MLLMs) can memorize and disclose sensitive information, motivating machine unlearning to remove designated knowledge while preserving model utilit…
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
Chenhan Jin, Shengze Xu, Qingsong Wang +3
Data pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined pruning method while striving f…
Efficient Private SCO for Heavy-Tailed Data via Averaged Clipping
Chenhan Jin, Kaiwen Zhou, Bo Han +2
We consider stochastic convex optimization for heavy-tailed data with the guarantee of being differentially private (DP). Most prior works on differentially private stochastic conv…
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms
Binghui Xie, Chenhan Jin, Kaiwen Zhou +2
Stochastic variance reduced methods have shown strong performance in solving finite-sum problems. However, these methods usually require the users to manually tune the step-size, w…