36 citations · 77 across the 14 of their papers we have counts for
14 papers
Kimi K3: Open Frontier Intelligence
Kimi Team, Tongtong Bai, Yifan Bai +398
We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is…
Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
Pin-Yu Chen, Han Shen, Payel Das +1
Fine-tuning Large Language Models (LLMs) on some task-specific datasets has been a primary use of LLMs. However, it has been empirically observed that this approach to enhancing ca…
SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
Han Shen, Pin-Yu Chen, Payel Das +1
Fine-tuning on task-specific data to boost downstream performance is a crucial step for leveraging Large Language Models (LLMs). However, previous studies have demonstrated that fi…
Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
Heshan Fernando, Han Shen, Parikshit Ram +4
The post-training of LLMs, which typically consists of the supervised fine-tuning (SFT) stage and the preference learning stage (RLHF or DPO), is crucial to effective and safe LLM…
Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
Han Shen, Zhuoran Yang, Tianyi Chen
Bilevel optimization has been recently applied to many machine learning tasks. However, their applications have been restricted to the supervised learning setting, where static obj…
Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization
A F M Saif, Xiaodong Cui, Han Shen +3
In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term {bi-level joint…
Differentially Private Non-convex Learning for Multi-layer Neural Networks
Hanpu Shen, Cheng-Long Wang, Zihang Xiang +2
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node. In the first part,…
On Penalty-based Bilevel Gradient Descent Method
Han Shen, Quan Xiao, Tianyi Chen
Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, met…
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization
Quan Xiao, Han Shen, Wotao Yin +1
Stochastic bilevel optimization, which captures the inherent nested structure of machine learning problems, is gaining popularity in many recent applications. Existing works on bil…
Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
Heshan Fernando, Han Shen, Miao Liu +3
Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance met…
A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences
Han Shen, Tianyi Chen
Stochastic approximation (SA) with multiple coupled sequences has found broad applications in machine learning such as bilevel learning and reinforcement learning (RL). In this pap…
Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup
Han Shen, Kaiqing Zhang, Mingyi Hong +1
Asynchronous and parallel implementation of standard reinforcement learning (RL) algorithms is a key enabler of the tremendous success of modern RL. Among many asynchronous RL algo…
Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation
Zhaoxian Wu, Han Shen, Tianyi Chen +1
This paper considers the policy evaluation problem in a multi-agent reinforcement learning (MARL) environment over decentralized and directed networks. The focus is on decentralize…
Adaptive Temporal Difference Learning with Linear Function Approximation
Tao Sun, Han Shen, Tianyi Chen +1
This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD() is ve…