13 papers · 1 filter
Personalized Federated Direct Preference Optimization via LoRA Adapters
Zhanhong Jiang, Soumik Sarkar
Federated Direct Preference Optimization (FedDPO) enables privacy-preserving alignment of language models across distributed agents without centralizing sensitive preference data.…
Continuous-Time Probabilistic Correctors for Uncertainty-Aware Physics-Based Spacecraft Trajectory Forecasting
Muhammad Bilal Shahid, Zhanhong Jiang, Soumik Sarkar +1
Long-horizon spacecraft trajectory forecasting suffers from error accumulation due to the absence of corrective observations in the forecast regime, making reliable uncertainty est…
TabQL: In-Context Q-Learning with Tabular Foundation Models
Qisai Liu, Zhanhong Jiang, Timilehin Ayanlade +4
We propose Tabular Q-Learning (TabQL), a reinforcement learning framework that replaces the conventional parametric Q-network in Deep Q-Learning (DQN) with a tabular foundation mod…
COOPO: Cyclic Offline-Online Policy Optimization Algorithm
Qisai Liu, Zhanhong Jiang, Joshua Russell Waite +3
Offline reinforcement learning struggles with distributional shift and constrained performance due to static dataset limitations, while online RL demands prohibitive environment in…
ADKO: Agentic Decentralized Knowledge Optimization
Lucas Nerone Rillo, Zhanhong Jiang, Nastaran Saadati +4
We present Agentic Decentralized Knowledge Optimization (ADKO), a framework for collaborative black-box optimization across autonomous agents that achieves sample efficiency, priva…
Neural CDEs as Correctors for Learned Time Series Models
Muhammad Bilal Shahid, Zhanhong Jiang, Prajwal Koirala +2
Learned time-series models, whether continuous or discrete, are widely used for forecasting the states of dynamical systems but suffer from error accumulation in multi-step forecas…
Balancing Utility and Privacy: Dynamically Private SGD with Random Projection
Zhanhong Jiang, Md Zahid Hasan, Nastaran Saadati +3
Stochastic optimization is a pivotal enabler in modern machine learning, producing effective models for various tasks. However, several existing works have shown that model paramet…
DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
Nastaran Saadati, Zhanhong Jiang, Joshua R. Waite +4
Low-Rank Adaptation (LoRA) has emerged as one of the most effective, computationally tractable fine-tuning approaches for training Vision-Language Models (VLMs) and Large Language…
Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion
Qisai Liu, Zhanhong Jiang, Joshua R. Waite +3
Sequence modeling is a critical yet challenging task with wide-ranging applications, especially in time series forecasting for domains like weather prediction, temperature monitori…
Enhancing PPO with Trajectory-Aware Hybrid Policies
Qisai Liu, Zhanhong Jiang, Hsin-Jung Yang +3
Proximal policy optimization (PPO) is one of the most popular state-of-the-art on-policy algorithms that has become a standard baseline in modern reinforcement learning with applic…
FAWAC: Feasibility Informed Advantage Weighted Regression for Persistent Safety in Offline Reinforcement Learning
Prajwal Koirala, Zhanhong Jiang, Soumik Sarkar +1
Safe offline reinforcement learning aims to learn policies that maximize cumulative rewards while adhering to safety constraints, using only offline data for training. A key challe…
Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
Prajwal Koirala, Zhanhong Jiang, Soumik Sarkar +1
In safe offline reinforcement learning (RL), the objective is to develop a policy that maximizes cumulative rewards while strictly adhering to safety constraints, utilizing only of…
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models
Nastaran Saadati, Minh Pham, Nasla Saleem +5
Recent advances in decentralized deep learning algorithms have demonstrated cutting-edge performance on various tasks with large pre-trained models. However, a pivotal prerequisite…