7 citations · 9 across the 14 of their papers we have counts for
14 papers
Aligning Large Language Models with Human Feedback: Mathematical Foundations and Algorithm Design
Siliang Zeng, Luca Viano, Chenliang Li +6
This article provides an introduction to the mathematical foundations and algorithmic frameworks used to align Large Language Models (LLMs) with human intentions, preferences, and…
On Second-Order Methods for Bilevel Optimization
Jiawen Bi, Jiaxiang Li, Mingyi Hong +1
Bilevel optimization is an indispensable modeling tool for modern machine learning and engineering design. However, the theory and practice for finding second order stationary poin…
Action-Effect Memory Pretraining for Robot Manipulation
Yijing Zhou, Qiwei Liang, Sitong Zhuang +5
We present AEM, an Action-Effect Memory pretraining framework for robot manipulation that learns compact temporal representations from vision-action history. Unlike prior robot rep…
A Framework for Quantifying How Pre-Training and Context Benefit In-Context Learning
Bingqing Song, Jiaxiang Li, Rong Wang +2
Pre-trained large language models have demonstrated a strong ability to learn from context, known as in-context learning (ICL). Despite a surge of recent applications that leverage…
ADARL: Adaptive Low-Rank Structures for Robust Policy Learning under Uncertainty
Chenliang Li, Junyu Leng, Jiaxiang Li +4
Robust reinforcement learning (Robust RL) seeks to handle epistemic uncertainty in environment dynamics, but existing approaches often rely on nested min--max optimization, which i…
Muon Outperforms Adam in Tail-End Associative Memory Learning
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +6
The Muon optimizer is consistently faster than Adam in training Large Language Models (LLMs), yet the mechanism underlying its success remains unclear. This paper demystifies this…
A Correspondence-Driven Approach for Bilevel Decision-making with Nonconvex Lower-Level Problems
Xiaotian Jiang, Jiaxiang Li, Jiawen Bi +2
We consider bilevel optimization problems with general nonconvex lower-level objectives and show that the classical hyperfunction-based formulation is unsettled, since the global m…
Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach
Xinnan Zhang, Chenliang Li, Siliang Zeng +6
Aligning large language models (LLMs) with human preferences usually requires fine-tuning methods such as RLHF and DPO. These methods directly optimize the model parameters, so the…
Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
Athanasios Glentis, Jiaxiang Li, Andi Han +1
Training large language models (LLMs) relies on adaptive optimizers such as Adam, which introduce extra operations and require significantly more memory to maintain first- and seco…
Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking
Athanasios Glentis, Jiaxiang Li, Qiulin Shang +4
Fueled by their remarkable ability to tackle diverse tasks across multiple domains, large language models (LLMs) have grown at an unprecedented rate, with some recent models contai…
A Note on the Convergence of Muon
Jiaxiang Li, Mingyi Hong
In this note, we inspect the convergence of a new optimizer for pretraining LLMs, namely the Muon optimizer. Such an optimizer is closely related to a specialized steepest descent…
A Barrier Function Approach for Bilevel Optimization with Coupled Lower-Level Constraints: Formulation, Approximation and Algorithms
Xiaotian Jiang, Jiaxiang Li, Mingyi Hong +1
In this paper, we consider bilevel optimization problem where the lower-level has coupled constraints, i.e. the constraints depend both on the upper- and lower-level variables. In…
Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment
Chenliang Li, Siliang Zeng, Zeyi Liao +4
Aligning human preference and value is an important requirement for building contemporary foundation models and embodied AI. However, popular approaches such as reinforcement learn…
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
Yihua Zhang, Pingzhi Li, Junyuan Hong +10
In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has become s…