Publications (33)
Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data
Xi Yan, David Acuna, Sanja Fidler
Transfer learning has proven to be a successful technique to train deep learning models in the domains where little training data is available. The dominant approach is to pretrain…
Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
David Acuna, Ximing Lu, Jaehun Jung +4
Recent research in vision-language models (VLMs) has centered around the possibility of equipping them with implicit long-form chain-of-thought reasoning -- akin to the success obs…
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Jonathan Tremblay, Aayush Prakash, David Acuna +7
We present a system for training deep neural networks for object detection using synthetic images. To handle the variability in real-world data, the system relies upon the techniqu…
RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting
Ashkan Mirzaei, Riccardo De Lutio, Seung Wook Kim +5
Neural reconstruction approaches are rapidly emerging as the preferred representation for 3D scenes, but their limited editability is still posing a challenge. In this work, we pro…
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks
Or Litany, Haggai Maron, David Acuna +3
Standard Federated Learning (FL) techniques are limited to clients with identical network architectures. This restricts potential use-cases like cross-platform training or inter-or…
Complex Momentum for Optimization in Games
Jonathan Lorraine, David Acuna, Paul Vicol +1
We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving conv…
Meta-Sim: Learning to Generate Synthetic Datasets
Amlan Kar, Aayush Prakash, Ming-Yu Liu +6
Training models to high-end performance requires availability of large labeled datasets, which are expensive to get. The goal of our work is to automatically synthesize labeled dat…
Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Viraj Prabhu, David Acuna, Andrew Liao +5
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domai…
LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception
Yuan-Hong Liao, Sven Elflein, Liu He +4
Recent reasoning models through test-time scaling have demonstrated that long chain-of-thoughts can unlock substantial performance boosts in hard reasoning tasks such as math and c…
How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
Bosung Kim, Ruiyi Wang, David Acuna +5
Scaling robot policy learning is bottlenecked by the cost of collecting demonstrations, while language annotations for existing demonstrations are comparatively cheap. We study lan…
Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction
Yong Lin, Shange Tang, Bohan Lyu +17
We introduce Goedel-Prover-V2, a series of open-source language models that set a new state-of-the-art in automated theorem proving. Built on the standard expert iteration and rein…
Domain Adversarial Training: A Game Perspective
David Acuna, Marc T Law, Guojun Zhang +1
The dominant line of work in domain adaptation has focused on learning invariant representations using domain-adversarial training. In this paper, we interpret this approach from a…
Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves?
Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler +1
Enhancing semantic grounding abilities in Vision-Language Models (VLMs) often involves collecting domain-specific training data, refining the network architectures, or modifying th…
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Jaehun Jung, Seungju Han, Ximing Lu +7
Effective generalization in language models depends critically on the diversity of their training data. Yet existing diversity metrics often fall short of this goal, relying on sur…
Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text
Ximing Lu, David Acuna, Jaehun Jung +12
Reinforcement Learning with Verifiable Rewards (RLVR) has become a cornerstone for unlocking complex reasoning in Large Language Models (LLMs). Yet, scaling up RL is bottlenecked b…
Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale
David Acuna, Chao-Han Huck Yang, Yuntian Deng +6
Despite rapid progress, multimodal reasoning still lacks a systematic approach to synthesize large-scale vision-centric datasets beyond visual math. We introduce a framework able t…
Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
Towaki Takikawa, David Acuna, Varun Jampani +1
Current state-of-the-art methods for image segmentation form a dense image representation where the color, shape and texture information are all processed together inside a deep CN…
Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion
Zian Wang, Wenzheng Chen, David Acuna +2
We consider the challenging problem of outdoor lighting estimation for the goal of photorealistic virtual object insertion into photographs. Existing works on outdoor lighting esti…
Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
Brandon Cui, Ximing Lu, Jaehun Jung +7
We tackle the question of how to scale more efficiently across the many, ever-growing stages of current LLM training pipelines. Our guiding intuition stems from the fact that the d…
DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
Jaehun Jung, Hyunwoo Kim, Brandon Cui +4
Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically chosen via simple heuristics…
f-Domain-Adversarial Learning: Theory and Algorithms
David Acuna, Guojun Zhang, Marc T. Law +1
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labele…
Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler +1
Despite recent advances demonstrating vision-language models' (VLMs) abilities to describe complex relationships in images using natural language, their capability to quantitativel…
DreamTeacher: Pretraining Image Backbones with Deep Generative Models
Daiqing Li, Huan Ling, Amlan Kar +5
In this work, we introduce a self-supervised feature representation learning framework DreamTeacher that utilizes generative networks for pre-training downstream image backbones. W…
Neural Turtle Graphics for Modeling City Road Layouts
Hang Chu, Daiqing Li, David Acuna +6
We propose Neural Turtle Graphics (NTG), a novel generative model for spatial graphs, and demonstrate its applications in modeling city road layouts. Specifically, we represent the…
Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotations
David Acuna, Amlan Kar, Sanja Fidler
We tackle the problem of semantic boundary prediction, which aims to identify pixels that belong to object(class) boundaries. We notice that relevant datasets consist of a signific…
Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation
David Acuna, Jonah Philion, Sanja Fidler
Autonomous driving relies on a huge volume of real-world data to be labeled to high precision. Alternative solutions seek to exploit driving simulators that can generate large amou…
Scalable Neural Data Server: A Data Recommender for Transfer Learning
Tianshi Cao, Sasha Doubov, David Acuna +1
Absence of large-scale labeled data in the practitioner's target domain can be a bottleneck to applying machine learning algorithms in practice. Transfer learning is a popular stra…
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
David Acuna, Huan Ling, Amlan Kar +1
Manually labeling datasets with object masks is extremely time consuming. In this work, we follow the idea of Polygon-RNN to produce polygonal annotations of objects interactively…
How Much More Data Do I Need? Estimating Requirements for Downstream Tasks
Rafid Mahmood, James Lucas, David Acuna +6
Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance? This question is of critical importance…
Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch
Hyunwoo Kim, Niloofar Mireshghallah, Michael Duan +11
Research involving privacy-sensitive data has always been constrained by data scarcity, standing in sharp contrast to other areas that have benefited from data scaling. This challe…
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Aayush Prakash, Shaad Boochoon, Mark Brophy +5
We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure and context of the scene. In contrast to DR, which pl…
Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering
Ruofan Liang, Zan Gojcic, Merlin Nimier-David +4
The correct insertion of virtual objects in images of real-world scenes requires a deep understanding of the scene's lighting, geometry and materials, as well as the image formatio…
Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning
Ximing Lu, Seungju Han, David Acuna +8
Large reasoning models exhibit remarkable reasoning capabilities via long, elaborate reasoning trajectories. Supervised fine-tuning on such reasoning traces, also known as distilla…