348 citations · 1.5k across the 43 of their papers we have counts for
49 papers
INVE: Interactive Neural Video Editing
Jiahui Huang, Leonid Sigal, Kwang Moo Yi +2
We present Interactive Neural Video Editing (INVE), a real-time video editing solution, which can assist the video editing process by consistently propagating sparse frame edits to…
StarUnLink: identifying and mitigating signals from communications satellites in stellar spectral surveys
Spencer Bialek, Sara Lucatello, Sebastien Fabbro +2
A relatively new concern for the forthcoming massive spectroscopic sky surveys is the impact of contamination from low earth orbit satellites. Several hundred thousand of these sat…
PPDONet: Deep Operator Networks for Fast Prediction of Steady-State Solutions in Disk-Planet Systems
Shunyuan Mao, Ruobing Dong, Lu Lu +3
We develop a tool, which we name Protoplanetary Disk Operator Network (PPDONet), that can predict the solution of disk-planet interactions in protoplanetary disks in real-time. We…
BlendFields: Few-Shot Example-Driven Facial Modeling
Kacper Kania, Stephan J. Garbin, Andrea Tagliasacchi +5
Generating faithful visualizations of human faces requires capturing both coarse and fine-level details of the face geometry and appearance. Existing methods are either data-driven…
Unsupervised Semantic Correspondence Using Stable Diffusion
Eric Hedlin, Gopal Sharma, Shweta Mahajan +4
Text-to-image diffusion models are now capable of generating images that are often indistinguishable from real images. To generate such images, these models must understand the sem…
CN-DHF: Compact Neural Double Height-Field Representations of 3D Shapes
Eric Hedlin, Jinfan Yang, Nicholas Vining +2
We introduce CN-DHF (Compact Neural Double-Height-Field), a novel hybrid neural implicit 3D shape representation that is dramatically more compact than the current state of the art…
Pointersect: Neural Rendering with Cloud-Ray Intersection
Jen-Hao Rick Chang, Wei-Yu Chen, Anurag Ranjan +2
We propose a novel method that renders point clouds as if they are surfaces. The proposed method is differentiable and requires no scene-specific optimization. This unique capabili…
FaceLit: Neural 3D Relightable Faces
Anurag Ranjan, Kwang Moo Yi, Jen-Hao Rick Chang +1
We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D image…
Neural Fourier Filter Bank
Zhijie Wu, Yuhe Jin, Kwang Moo Yi
We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequ…
Bootstrapping Human Optical Flow and Pose
Aritro Roy Arko, James J. Little, Kwang Moo Yi
We propose a bootstrapping framework to enhance human optical flow and pose. We show that, for videos involving humans in scenes, we can improve both the optical flow and the pose…
Attention Beats Concatenation for Conditioning Neural Fields
Daniel Rebain, Mark J. Matthews, Kwang Moo Yi +3
Neural fields model signals by mapping coordinate inputs to sampled values. They are becoming an increasingly important backbone architecture across many fields from vision and gra…
Estimating Visual Information From Audio Through Manifold Learning
Fabrizio Pedersoli, Dryden Wiebe, Amin Banitalebi +3
We propose a new framework for extracting visual information about a scene only using audio signals. Audio-based methods can overcome some of the limitations of vision-based method…
NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Clouds
Weiwei Sun, Daniel Rebain, Renjie Liao +4
We introduce a method for instance proposal generation for 3D point clouds. Existing techniques typically directly regress proposals in a single feed-forward step, leading to inacc…
TUSK: Task-Agnostic Unsupervised Keypoints
Yuhe Jin, Weiwei Sun, Jan Hosang +2
Existing unsupervised methods for keypoint learning rely heavily on the assumption that a specific keypoint type (e.g. elbow, digit, abstract geometric shape) appears only once in…
FlowNet-PET: Unsupervised Learning to Perform Respiratory Motion Correction in PET Imaging
Teaghan O'Briain, Carlos Uribe, Kwang Moo Yi +3
To correct for respiratory motion in PET imaging, an interpretable and unsupervised deep learning technique, FlowNet-PET, was constructed. The network was trained to predict the op…
A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset
Eric Hedlin, Helge Rhodin, Kwang Moo Yi
Many human pose estimation methods estimate Skinned Multi-Person Linear (SMPL) models and regress the human joints from these SMPL estimates. In this work, we show that the most wi…
Kubric: A scalable dataset generator
Klaus Greff, Francois Belletti, Lucas Beyer +32
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and trainin…
NeuMan: Neural Human Radiance Field from a Single Video
Wei Jiang, Kwang Moo Yi, Golnoosh Samei +2
Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences. We propose a novel framework to reconstruct the human and the scene that ca…
Repurposing Existing Deep Networks for Caption and Aesthetic-Guided Image Cropping
Nora Horanyi, Kedi Xia, Kwang Moo Yi +3
We propose a novel optimization framework that crops a given image based on user description and aesthetics. Unlike existing image cropping methods, where one typically trains a de…
Layered Controllable Video Generation
Jiahui Huang, Yuhe Jin, Kwang Moo Yi +1
We introduce layered controllable video generation, where we, without any supervision, decompose the initial frame of a video into foreground and background layers, with which the…
LOLNeRF: Learn from One Look
Daniel Rebain, Mark Matthews, Kwang Moo Yi +2
We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object. While generating realistic i…
Deep Medial Fields
Daniel Rebain, Ke Li, Vincent Sitzmann +3
Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form.…
Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG
Robert W. Bickley, Connor Bottrell, Maan H. Hani +6
The Canada-France Imaging Survey (CFIS) will consist of deep, high-resolution r-band imaging over ~5000 square degrees of the sky, representing a first-rate opportunity to identify…
COTR: Correspondence Transformer for Matching Across Images
Wei Jiang, Eduard Trulls, Jan Hosang +2
We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence…