8 citations · 8 across the 15 of their papers we have counts for
9 papers · 1 filter
Shift-aware score transport for privacy auditing without retraining
Yaohong Yang, Gauri Pradhan, Qi Chen +2
Privacy auditing empirically assesses privacy leakage by using membership inference attacks to establish lower bounds on privacy loss. We consider an observational auditing setting…
The Hidden Ratio in Adam: Stable Structure, Compression, and Sign Dynamics
Yihe Zhou, Tongtian Zhu, Yingxiao Huo +4
Adam is the default optimizer for training modern deep neural networks, yet its adaptive behavior remains poorly understood due to the complex interaction between its first- and se…
Human-Machine Collaboration on Generative Meta-Learning: Model and Algorithm
Midhun Parakkal Unni, Samuel Kaski
Generalizing machine learning models to environments that differ from their training distribution remains a critical hurdle, particularly when data from the target domain is entire…
Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
Yingxiao Huo, Satya Prakash Dash, Radu Stoican +2
Natural gradients have long been studied in deep reinforcement learning due to their fast convergence properties and covariant weight updates. However, computing natural gradients…
Gradient Regularized Natural Gradients
Satya Prakash Dash, Hossein Abdi, Wei Pan +2
Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initi…
Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations
Victor M. Yeom-Song, Severi Rissanen, Arno Solin +2
Diffusion models have become a powerful generative prior for solutions of partial differential equations (PDEs). Existing approaches enforce physical constraints either by adding t…
Rethinking Inter-LoRA Orthogonality in Adapter Merging: Insights from Orthogonal Monte Carlo Dropout
Andi Zhang, Xuan Ding, Haofan Wang +2
We propose Orthogonal Monte Carlo Dropout, a mechanism that enforces strict orthogonality when combining sparse semantic vectors without extra time complexity. Low-Rank Adaptation…
An Interactive Framework for Finding the Optimal Trade-off in Differential Privacy
Yaohong Yang, Aki Rehn, Sammie Katt +2
Differential privacy (DP) is the standard for privacy-preserving analysis, and introduces a fundamental trade-off between privacy guarantees and model performance. Selecting the op…
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
Hossein Abdi, Mingfei Sun, Andi Zhang +2
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…