565 citations · 905 across the 19 of their papers we have counts for
20 papers
Video models are zero-shot learners and reasoners
Thaddäus Wiedemer, Yuxuan Li, Paul Vicol +6
The remarkable zero-shot capabilities of Large Language Models (LLMs) have propelled natural language processing from task-specific models to unified, generalist foundation models.…
QDM: Quadtree-Based Region-Adaptive Sparse Diffusion Models for Efficient Image Super-Resolution
Donglin Yang, Paul Vicol, Xiaojuan Qi +2
Deep learning-based super-resolution (SR) methods often perform pixel-wise computations uniformly across entire images, even in homogeneous regions where high-resolution refinement…
Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation
Xiaoying Xing, Avinab Saha, Junfeng He +9
Text-to-image (T2I) generation has made significant advances in recent years, but challenges still remain in the generation of perceptual artifacts, misalignment with complex promp…
Directly Fine-Tuning Diffusion Models on Differentiable Rewards
Kevin Clark, Paul Vicol, Kevin Swersky +1
We present Direct Reward Fine-Tuning (DRaFT), a simple and effective method for fine-tuning diffusion models to maximize differentiable reward functions, such as scores from human…
Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single
Paul Vicol, Zico Kolter, Kevin Swersky
We propose an evolution strategies-based algorithm for estimating gradients in unrolled computation graphs, called ES-Single. Similarly to the recently-proposed Persistent Evolutio…
On Implicit Bias in Overparameterized Bilevel Optimization
Paul Vicol, Jonathan Lorraine, Fabian Pedregosa +2
Many problems in machine learning involve bilevel optimization (BLO), including hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems consist of tw…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Amortized Proximal Optimization
Juhan Bae, Paul Vicol, Jeff Z. HaoChen +1
We propose a framework for online meta-optimization of parameters that govern optimization, called Amortized Proximal Optimization (APO). We first interpret various existing neural…
Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies
Paul Vicol, Luke Metz, Jascha Sohl-Dickstein
Unrolled computation graphs arise in many scenarios, including training RNNs, tuning hyperparameters through unrolled optimization, and training learned optimizers. Current approac…
Lyapunov Exponents for Diversity in Differentiable Games
Jonathan Lorraine, Paul Vicol, Jack Parker-Holder +3
Ridge Rider (RR) is an algorithm for finding diverse solutions to optimization problems by following eigenvectors of the Hessian ("ridges"). RR is designed for conservative gradien…
Disentanglement and Generalization Under Correlation Shifts
Christina M. Funke, Paul Vicol, Kuan-Chieh Wang +3
Correlations between factors of variation are prevalent in real-world data. Exploiting such correlations may increase predictive performance on noisy data; however, often correlati…
Complex Momentum for Optimization in Games
Jonathan Lorraine, David Acuna, Paul Vicol +1
We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving conv…
Cost-Efficient Online Hyperparameter Optimization
Jingkang Wang, Mengye Ren, Ilija Bogunovic +2
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters. However, these online HPO algorith…
Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
Jens Behrmann, Paul Vicol, Kuan-Chieh Wang +2
Invertible neural networks (INNs) have been used to design generative models, implement memory-saving gradient computation, and solve inverse problems. In this work, we show that c…
Optimizing Millions of Hyperparameters by Implicit Differentiation
Jonathan Lorraine, Paul Vicol, David Duvenaud
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.…
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
Matthew MacKay, Paul Vicol, Jon Lorraine +2
Hyperparameter optimization can be formulated as a bilevel optimization problem, where the optimal parameters on the training set depend on the hyperparameters. We aim to adapt reg…
Reversible Recurrent Neural Networks
Matthew MacKay, Paul Vicol, Jimmy Ba +1
Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which…
Adversarial Distillation of Bayesian Neural Network Posteriors
Kuan-Chieh Wang, Paul Vicol, James Lucas +3
Bayesian neural networks (BNNs) allow us to reason about uncertainty in a principled way. Stochastic Gradient Langevin Dynamics (SGLD) enables efficient BNN learning by drawing sam…
Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
Yeming Wen, Paul Vicol, Jimmy Ba +2
Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies. Un…
MovieGraphs: Towards Understanding Human-Centric Situations from Videos
Paul Vicol, Makarand Tapaswi, Lluis Castrejon +1
There is growing interest in artificial intelligence to build socially intelligent robots. This requires machines to have the ability to "read" people's emotions, motivations, and…