204 citations · 711 across the 47 of their papers we have counts for
44 papers · 1 filter
Everybody Tracking Every Body
Daeyun Shin, Yunhan Zhao, Shu Kong +2
We address the problem of 3D body pose estimation of multiple interacting people from their egocentric views with centralized coordination. Each individual wears a camera recording…
Camera Control for Text-to-Image Generation via Learning Viewpoint Tokens
Xinxuan Lu, Charless Fowlkes, Alexander C. Berg
Current text-to-image models struggle to provide precise camera control using natural language alone. In this work, we present a framework for precise camera control with global sc…
CriSp: Leveraging Tread Depth Maps for Enhanced Crime-Scene Shoeprint Matching
Samia Shafique, Shu Kong, Charless Fowlkes
Shoeprints are a common type of evidence found at crime scenes and are used regularly in forensic investigations. However, existing methods cannot effectively employ deep learning…
Instance Tracking in 3D Scenes from Egocentric Videos
Yunhan Zhao, Haoyu Ma, Shu Kong +1
Egocentric sensors such as AR/VR devices capture human-object interactions and offer the potential to provide task-assistance by recalling 3D locations of objects of interest in th…
Creating a Forensic Database of Shoeprints from Online Shoe Tread Photos
Samia Shafique, Bailey Kong, Shu Kong +1
Shoe tread impressions are one of the most common types of evidence left at crime scenes. However, the utility of such evidence is limited by the lack of databases of footwear prin…
The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation
Zhe Wang, Jimei Yang, Charless Fowlkes
Nonparametric based methods have recently shown promising results in reconstructing human bodies from monocular images while model-based methods can help correct these estimates an…
GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding
Yunhan Zhao, Connelly Barnes, Yuqian Zhou +3
Reference-guided image inpainting restores image pixels by leveraging the content from another single reference image. The primary challenge is how to precisely place the pixels fr…
Representation Consolidation for Training Expert Students
Zhizhong Li, Avinash Ravichandran, Charless Fowlkes +3
Traditionally, distillation has been used to train a student model to emulate the input/output functionality of a teacher. A more useful goal than emulation, yet under-explored, is…
SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentation
Zhe Wang, Hao Chen, Xinyu Li +4
Temporal action segmentation is a task to classify each frame in the video with an action label. However, it is quite expensive to annotate every frame in a large corpus of videos…
A linearized framework and a new benchmark for model selection for fine-tuning
Aditya Deshpande, Alessandro Achille, Avinash Ravichandran +6
Fine-tuning from a collection of models pre-trained on different domains (a "model zoo") is emerging as a technique to improve test accuracy in the low-data regime. However, model…
Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
Yunhan Zhao, Shu Kong, Charless Fowlkes
Monocular depth predictors are typically trained on large-scale training sets which are naturally biased w.r.t the distribution of camera poses. As a result, trained predictors fai…
Weak Supervision and Referring Attention for Temporal-Textual Association Learning
Zhiyuan Fang, Shu Kong, Zhe Wang +2
A system capturing the association between video frames and textual queries offer great potential for better video analysis. However, training such a system in a fully supervised w…
Celeganser: Automated Analysis of Nematode Morphology and Age
Linfeng Wang, Shu Kong, Zachary Pincus +1
The nematode Caenorhabditis elegans (C. elegans) serves as an important model organism in a wide variety of biological studies. In this paper we introduce a pipeline for automated…
Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation
Zhe Wang, Daeyun Shin, Charless C. Fowlkes
Monocular estimation of 3d human pose has attracted increased attention with the availability of large ground-truth motion capture datasets. However, the diversity of training data…
Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation
Yunhan Zhao, Shu Kong, Daeyun Shin +1
Leveraging synthetically rendered data offers great potential to improve monocular depth estimation and other geometric estimation tasks, but closing the synthetic-real domain gap…
Weakly-supervised Action Localization with Background Modeling
Phuc Xuan Nguyen, Deva Ramanan, Charless C. Fowlkes
We describe a latent approach that learns to detect actions in long sequences given training videos with only whole-video class labels. Our approach makes use of two innovations to…
Geometric Pose Affordance: 3D Human Pose with Scene Constraints
Zhe Wang, Liyan Chen, Shaurya Rathore +2
Full 3D estimation of human pose from a single image remains a challenging task despite many recent advances. In this paper, we explore the hypothesis that strong prior information…
Multigrid Predictive Filter Flow for Unsupervised Learning on Videos
Shu Kong, Charless Fowlkes
We introduce multigrid Predictive Filter Flow (mgPFF), a framework for unsupervised learning on videos. The mgPFF takes as input a pair of frames and outputs per-pixel filters to w…
Modularized Textual Grounding for Counterfactual Resilience
Zhiyuan Fang, Shu Kong, Charless Fowlkes +1
Computer Vision applications often require a textual grounding module with precision, interpretability, and resilience to counterfactual inputs/queries. To achieve high grounding p…
3D Scene Reconstruction with Multi-layer Depth and Epipolar Transformers
Daeyun Shin, Zhile Ren, Erik B. Sudderth +1
We tackle the problem of automatically reconstructing a complete 3D model of a scene from a single RGB image. This challenging task requires inferring the shape of both visible and…
Active Testing: An Efficient and Robust Framework for Estimating Accuracy
Phuc Nguyen, Deva Ramanan, Charless Fowlkes
Much recent work on visual recognition aims to scale up learning to massive, noisily-annotated datasets. We address the problem of scaling- up the evaluation of such models to larg…
Fine-Grained Facial Expression Analysis Using Dimensional Emotion Model
Feng Zhou, Shu Kong, Charless Fowlkes +2
Automated facial expression analysis has a variety of applications in human-computer interaction. Traditional methods mainly analyze prototypical facial expressions of no more than…
Pixel-wise Attentional Gating for Parsimonious Pixel Labeling
Shu Kong, Charless Fowlkes
To achieve parsimonious inference in per-pixel labeling tasks with a limited computational budget, we propose a \emph{Pixel-wise Attentional Gating} unit (\emph{PAG}) that learns t…
Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction
Daeyun Shin, Charless C. Fowlkes, Derek Hoiem
The goal of this paper is to compare surface-based and volumetric 3D object shape representations, as well as viewer-centered and object-centered reference frames for single-view 3…