315 citations · 1.2k across the 43 of their papers we have counts for
55 papers
Training Generalist Navigation Policies via Action Self-Distillation
Bowen Feng, Keqin Wang, Yulong Yang +4
Language provides a natural interface for humans to specify and reason about goals, but training language-conditioned robotic policies and reasoning typically requires expensive ma…
EpicWorldModel: Exploration-driven Planning with Latent World Models
Bowen Feng, Julian Ost, May Mei +2
Latent world models based on Joint-Embedding Predictive Architecture (JEPA) are deterministic by design. While successful in fully observable scenarios, this paradigm breaks down w…
SCION: Scene Composition with Instanced Neural Primitives
William Koch, Amogh Joshi, Cyrus Vachha +2
Real-world scenes are compositional: bricks, blades of grass, pebbles, and tree leaves recur across human-built and natural environments. Existing neural scene representations mode…
In the Blink of an Eye: Event-based Emotion Recognition
Haiwei Zhang, Jiqing Zhang, Bo Dong +5
We introduce a wearable single-eye emotion recognition device and a real-time approach to recognizing emotions from partial observations of an emotion that is robust to changes in…
Kissing to Find a Match: Efficient Low-Rank Permutation Representation
Hannah Dröge, Zorah Lähner, Yuval Bahat +3
Permutation matrices play a key role in matching and assignment problems across the fields, especially in computer vision and robotics. However, memory for explicitly representing…
Thin On-Sensor Nanophotonic Array Cameras
Praneeth Chakravarthula, Jipeng Sun, Xiao Li +6
Today's commodity camera systems rely on compound optics to map light originating from the scene to positions on the sensor where it gets recorded as an image. To record images wit…
Spatially Varying Nanophotonic Neural Networks
Kaixuan Wei, Xiao Li, Johannes Froech +5
The explosive growth of computation and energy cost of artificial intelligence has spurred strong interests in new computing modalities as potential alternatives to conventional el…
Stochastic Light Field Holography
Florian Schiffers, Praneeth Chakravarthula, Nathan Matsuda +5
The Visual Turing Test is the ultimate goal to evaluate the realism of holographic displays. Previous studies have focused on addressing challenges such as limited étendue and imag…
Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues
Stefanie Walz, Mario Bijelic, Andrea Ramazzina +3
We propose Gated Stereo, a high-resolution and long-range depth estimation technique that operates on active gated stereo images. Using active and high dynamic range passive captur…
STrack: Self-supervised Tracking with Soft Assignment Flow
Fatemeh Azimi, Fahim Mannan, Felix Heide
In this work, we study self-supervised multiple object tracking without using any video-level association labels. We propose to cast the problem of multiple object tracking as lear…
ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering
Andrea Ramazzina, Mario Bijelic, Stefanie Walz +3
Vision in adverse weather conditions, whether it be snow, rain, or fog is challenging. In these scenarios, scattering and attenuation severly degrades image quality. Handling such…
Instance Segmentation in the Dark
Linwei Chen, Ying Fu, Kaixuan Wei +2
Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. I…
Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography
Ilya Chugunov, Yuxuan Zhang, Felix Heide
Modern mobile burst photography pipelines capture and merge a short sequence of frames to recover an enhanced image, but often disregard the 3D nature of the scene they capture, tr…
Neural Volume Super-Resolution
Yuval Bahat, Yuxuan Zhang, Hendrik Sommerhoff +2
Neural volumetric representations have become a widely adopted model for radiance fields in 3D scenes. These representations are fully implicit or hybrid function approximators of…
The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection
Geoffroi Côté, Fahim Mannan, Simon Thibault +2
Most camera lens systems are designed in isolation, separately from downstream computer vision methods. Recently, joint optimization approaches that design lenses alongside other c…
Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions
Gene Chou, Yuval Bahat, Felix Heide
Probabilistic diffusion models have achieved state-of-the-art results for image synthesis, inpainting, and text-to-image tasks. However, they are still in the early stages of gener…
Biologically Inspired Dynamic Thresholds for Spiking Neural Networks
Jianchuan Ding, Bo Dong, Felix Heide +4
The dynamic membrane potential threshold, as one of the essential properties of a biological neuron, is a spontaneous regulation mechanism that maintains neuronal homeostasis, i.e.…
GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions
Gene Chou, Ilya Chugunov, Felix Heide
We investigate the generalization capabilities of neural signed distance functions (SDFs) for learning 3D object representations for unseen and unlabeled point clouds. Existing met…
Differentiable Point-Based Radiance Fields for Efficient View Synthesis
Qiang Zhang, Seung-Hwan Baek, Szymon Rusinkiewicz +1
We propose a differentiable rendering algorithm for efficient novel view synthesis. By departing from volume-based representations in favor of a learned point representation, we im…
LiDAR Snowfall Simulation for Robust 3D Object Detection
Martin Hahner, Christos Sakaridis, Mario Bijelic +4
3D object detection is a central task for applications such as autonomous driving, in which the system needs to localize and classify surrounding traffic agents, even in the presen…
Pupil-aware Holography
Praneeth Chakravarthula, Seung-Hwan Baek, Florian Schiffers +7
Holographic displays promise to deliver unprecedented display capabilities in augmented reality applications, featuring a wide field of view, wide color gamut, spatial resolution,…
All-photon Polarimetric Time-of-Flight Imaging
Seung-Hwan Baek, Felix Heide
Time-of-flight (ToF) sensors provide an imaging modality fueling diverse applications, including LiDAR in autonomous driving, robotics, and augmented reality. Conventional ToF imag…
Gated2Gated: Self-Supervised Depth Estimation from Gated Images
Amanpreet Walia, Stefanie Walz, Mario Bijelic +5
Gated cameras hold promise as an alternative to scanning LiDAR sensors with high-resolution 3D depth that is robust to back-scatter in fog, snow, and rain. Instead of sequentially…
All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines
Yuxuan Zhang, Bo Dong, Felix Heide
Existing neural networks for computer vision tasks are vulnerable to adversarial attacks: adding imperceptible perturbations to the input images can fool these methods to make a fa…