841 citations · 2.6k across the 48 of their papers we have counts for
49 papers
A General Framework for Inference-time Scaling and Steering of Diffusion Models
Raghav Singhal, Zachary Horvitz, Ryan Teehan +4
Diffusion models produce impressive results in modalities ranging from images and video to protein design and text. However, generating samples with user-specified properties remai…
Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle
Hui Dai, Ryan Teehan, Mengye Ren
Many existing evaluation benchmarks for Large Language Models (LLMs) quickly become outdated due to the emergence of new models and training data. These benchmarks also fall short…
ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation
Jack Lu, Ryan Teehan, Mengye Ren
In this paper, we propose ProCreate, a simple and easy-to-implement method to improve sample diversity and creativity of diffusion-based image generative models and to prevent trai…
PooDLe: Pooled and dense self-supervised learning from naturalistic videos
Alex N. Wang, Christopher Hoang, Yuwen Xiong +2
Self-supervised learning has driven significant progress in learning from single-subject, iconic images. However, there are still unanswered questions about the use of minimally-cu…
Integrating Present and Past in Unsupervised Continual Learning
Yipeng Zhang, Laurent Charlin, Richard Zemel +1
We formulate a unifying framework for unsupervised continual learning (UCL), which disentangles learning objectives that are specific to the present and the past data, encompassing…
CoLLEGe: Concept Embedding Generation for Large Language Models
Ryan Teehan, Brenden Lake, Mengye Ren
Current language models are unable to quickly learn new concepts on the fly, often requiring a more involved finetuning process to learn robustly. Prompting in-context is not robus…
Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training
Yanlai Yang, Matt Jones, Michael C. Mozer +1
We explore the training dynamics of neural networks in a structured non-IID setting where documents are presented cyclically in a fixed, repeated sequence. Typically, networks suff…
Self-supervised learning of video representations from a child's perspective
A. Emin Orhan, Wentao Wang, Alex N. Wang +2
Children learn powerful internal models of the world around them from a few years of egocentric visual experience. Can such internal models be learned from a child's visual experie…
Learning and Forgetting Unsafe Examples in Large Language Models
Jiachen Zhao, Zhun Deng, David Madras +2
As the number of large language models (LLMs) released to the public grows, there is a pressing need to understand the safety implications associated with these models learning fro…
LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos
Ying Wang, Yanlai Yang, Mengye Ren
In this paper we introduce LifelongMemory, a new framework for accessing long-form egocentric videographic memory through natural language question answering and retrieval. Lifelon…
BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling
Yixuan Luo, Mengye Ren, Sai Qian Zhang
Like masked language modeling (MLM) in natural language processing, masked image modeling (MIM) aims to extract valuable insights from image patches to enhance the feature extracti…
Towards Unsupervised Object Detection From LiDAR Point Clouds
Lunjun Zhang, Anqi Joyce Yang, Yuwen Xiong +4
In this paper, we study the problem of unsupervised object detection from 3D point clouds in self-driving scenes. We present a simple yet effective method that exploits (i) point c…
Rethinking Closed-loop Training for Autonomous Driving
Chris Zhang, Runsheng Guo, Wenyuan Zeng +5
Recent advances in high-fidelity simulators have enabled closed-loop training of autonomous driving agents, potentially solving the distribution shift in training v.s. deployment a…
Gaussian-Bernoulli RBMs Without Tears
Renjie Liao, Simon Kornblith, Mengye Ren +2
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations. We propose a novel Gibbs-Langevin sampling alg…
Scaling Forward Gradient With Local Losses
Mengye Ren, Simon Kornblith, Renjie Liao +1
Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks. However, the standard for…
Learning to Reason With Relational Abstractions
Andrew J. Nam, Mengye Ren, Chelsea Finn +1
Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.…
Online Unsupervised Learning of Visual Representations and Categories
Mengye Ren, Tyler R. Scott, Michael L. Iuzzolino +2
Real world learning scenarios involve a nonstationary distribution of classes with sequential dependencies among the samples, in contrast to the standard machine learning formulati…
Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
Sean Segal, Nishanth Kumar, Sergio Casas +4
Self-driving vehicles must perceive and predict the future positions of nearby actors in order to avoid collisions and drive safely. A learned deep learning module is often respons…
Cost-Efficient Online Hyperparameter Optimization
Jingkang Wang, Mengye Ren, Ilija Bogunovic +2
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters. However, these online HPO algorith…
SceneGen: Learning to Generate Realistic Traffic Scenes
Shuhan Tan, Kelvin Wong, Shenlong Wang +3
We consider the problem of generating realistic traffic scenes automatically. Existing methods typically insert actors into the scene according to a set of hand-crafted heuristics…
Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving
James Tu, Huichen Li, Xinchen Yan +6
Modern self-driving perception systems have been shown to improve upon processing complementary inputs such as LiDAR with images. In isolation, 2D images have been found to be extr…
Adversarial Attacks On Multi-Agent Communication
James Tu, Tsunhsuan Wang, Jingkang Wang +3
Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems. Sharing information and distributi…
Self-Supervised Representation Learning from Flow Equivariance
Yuwen Xiong, Mengye Ren, Wenyuan Zeng +1
Self-supervised representation learning is able to learn semantically meaningful features; however, much of its recent success relies on multiple crops of an image with very few ob…
AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
Jingkang Wang, Ava Pun, James Tu +5
As self-driving systems become better, simulating scenarios where the autonomy stack may fail becomes more important. Traditionally, those scenarios are generated for a few scenes…