21 citations · 43 across the 30 of their papers we have counts for
12 papers · 1 filter
It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs
Sangwoo Park, Woongyeong Yeo, Seanie Lee +6
Contextual Integrity (CI) defines privacy not merely as keeping information hidden, but as governing information flows according to the norms of a given context. As large language…
FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
Seanie Lee, Sangwoo Park, Dong Bok Lee +5
Low-Rank Adaptation (LoRA), which introduces a product of two trainable low-rank matrices into frozen pre-trained weights, is widely used for efficient fine-tuning of language mode…
FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates
Sangwoo Park, Seanie Lee, Byungjoo Kim +1
Federated Learning (FL) is a widely used framework for training models in a decentralized manner, ensuring that the central server does not have direct access to data from local cl…
Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
Brian Bartoldson, Siddarth Venkatraman, James Diffenderfer +7
Reinforcement learning (RL) is a critical component of large language model (LLM) post-training. However, on-policy algorithms used for post-training are not naturally robust to a…
Optimized Speculative Sampling for GPU Hardware Accelerators
Dominik Wagner, Seanie Lee, Ilja Baumann +3
In this work, we optimize speculative sampling for parallel hardware accelerators to improve sampling speed. We notice that substantial portions of the intermediate matrices necess…
Self-Supervised Dataset Distillation for Transfer Learning
Dong Bok Lee, Seanie Lee, Joonho Ko +3
Dataset distillation methods have achieved remarkable success in distilling a large dataset into a small set of representative samples. However, they are not designed to produce a…
Drug Discovery with Dynamic Goal-aware Fragments
Seul Lee, Seanie Lee, Kenji Kawaguchi +1
Fragment-based drug discovery is an effective strategy for discovering drug candidates in the vast chemical space, and has been widely employed in molecular generative models. Howe…
DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models
Sohyun An, Hayeon Lee, Jaehyeong Jo +2
Existing NAS methods suffer from either an excessive amount of time for repetitive sampling and training of many task-irrelevant architectures. To tackle such limitations of existi…
On Divergence Measures for Bayesian Pseudocoresets
Balhae Kim, Jungwon Choi, Seanie Lee +3
A Bayesian pseudocoreset is a small synthetic dataset for which the posterior over parameters approximates that of the original dataset. While promising, the scalability of Bayesia…
Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
Jeffrey Willette, Seanie Lee, Bruno Andreis +3
Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaran…
Set-based Meta-Interpolation for Few-Task Meta-Learning
Seanie Lee, Bruno Andreis, Kenji Kawaguchi +2
Meta-learning approaches enable machine learning systems to adapt to new tasks given few examples by leveraging knowledge from related tasks. However, a large number of meta-traini…
Set Based Stochastic Subsampling
Bruno Andreis, Seanie Lee, A. Tuan Nguyen +3
Deep models are designed to operate on huge volumes of high dimensional data such as images. In order to reduce the volume of data these models must process, we propose a set-based…