12 papers
Noise-Space Reward Alignment for Few-Step Generative Models
Woocheol Shin, Taeyoung Yun, Jiwon Sung +7
Few-step generative models produce high-quality samples in one or a few evaluations. However, steering them toward desired properties remains difficult, especially when only black-…
FestDPO: Few-step Generator Alignment with Direct Preference Optimization
Jaewoo Lee, Kyuil Sim, Hyeongyu Kang +3
Few-step generative models can generate high-fidelity samples within a few function evaluations. Despite this efficiency, generated samples may not exhibit desirable properties. Wh…
Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference
Jaewoo Lee, Hyeongyu Kang, Dohyun Kim +9
Aligning a few-step generative model is challenging, since existing alignment frameworks typically rely on restrictive assumptions: a tractable likelihood, a specific ODE/SDE solve…
Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization
Taeyoung Yun, Woocheol Shin, Inhyuck Song +2
Gaussian Process (GP) kernels are central to Bayesian optimization (BO), yet designing effective kernels for high-dimensional problems still relies on extensive manual engineering.…
Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning
Chihyeon Song, Jaewoo Lee, Jinkyoo Park
Offline-to-Online Reinforcement Learning (O2O RL) faces a critical dilemma in balancing the use of a fixed offline dataset with newly collected online experiences. Standard methods…
Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Function
Hyeongyu Kang, Jaewoo Lee, Woocheol Shin +2
Diffusion models excel at generating high-likelihood samples but often require alignment with downstream objectives. Existing fine-tuning methods for diffusion models significantly…
PRInTS: Reward Modeling for Long-Horizon Information Seeking
Jaewoo Lee, Archiki Prasad, Justin Chih-Yao Chen +3
Information-seeking is a core capability for AI agents, requiring them to gather and reason over tool-generated information across long trajectories. However, such multi-step infor…
Diffusion Alignment as Variational Expectation-Maximization
Jaewoo Lee, Minsu Kim, Sanghyeok Choi +7
Diffusion alignment aims to optimize diffusion models for the downstream objective. While existing methods based on reinforcement learning or direct backpropagation achieve conside…
TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models
Jaewoo Lee, Keyang Xuan, Chanakya Ekbote +3
Multimodal Large Language Models (MLLMs) have shown remarkable versatility in understanding diverse multimodal data and tasks. However, these capabilities come with an increased mo…
Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Taeyoung Yun, Kiyoung Om, Jaewoo Lee +2
Optimizing high-dimensional and complex black-box functions is crucial in numerous scientific applications. While Bayesian optimization (BO) is a powerful method for sample-efficie…
Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
Taeyoung Yun, Sujin Yun, Jaewoo Lee +1
Optimizing complex and high-dimensional black-box functions is ubiquitous in science and engineering fields. Unfortunately, the online evaluation of these functions is restricted d…
GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning
Jaewoo Lee, Sujin Yun, Taeyoung Yun +1
Offline Reinforcement Learning (Offline RL) presents challenges of learning effective decision-making policies from static datasets without any online interactions. Data augmentati…