86 citations · 543 across the 30 of their papers we have counts for
26 papers · 1 filter
Neural Design Network: Graphic Layout Generation with Constraints
Hsin-Ying Lee, Lu Jiang, Irfan Essa +4
Graphic design is essential for visual communication with layouts being fundamental to composing attractive designs. Layout generation differs from pixel-level image synthesis and…
Novel evaluation of surgical activity recognition models using task-based efficiency metrics
Aneeq Zia, Liheng Guo, Linlin Zhou +2
Purpose: Surgical task-based metrics (rather than entire procedure metrics) can be used to improve surgeon training and, ultimately, patient care through focused training intervent…
Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction
Steven Hickson, Karthik Raveendran, Alireza Fathi +2
We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These ins…
Embodied Question Answering in Photorealistic Environments with Point Cloud Perception
Erik Wijmans, Samyak Datta, Oleksandr Maksymets +6
To help bridge the gap between internet vision-style problems and the goal of vision for embodied perception we instantiate a large-scale navigation task -- Embodied Question Answe…
Audio-Visual Scene-Aware Dialog
Huda Alamri, Vincent Cartillier, Abhishek Das +9
We introduce the task of scene-aware dialog. Our goal is to generate a complete and natural response to a question about a scene, given video and audio of the scene and the history…
Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
Jonathan C Balloch, Varun Agrawal, Irfan Essa +1
Robot perception systems need to perform reliable image segmentation in real-time on noisy, raw perception data. State-of-the-art segmentation approaches use large CNN models and c…
Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action Recognition
Unaiza Ahsan, Rishi Madhok, Irfan Essa
We propose a self-supervised learning method to jointly reason about spatial and temporal context for video recognition. Recent self-supervised approaches have used spatial context…
Surgical Activity Recognition in Robot-Assisted Radical Prostatectomy using Deep Learning
Aneeq Zia, Andrew Hung, Irfan Essa +1
Adverse surgical outcomes are costly to patients and hospitals. Approaches to benchmark surgical care are often limited to gross measures across the entire procedure despite the pe…
Efficient Hierarchical Graph-Based Segmentation of RGBD Videos
Steven Hickson, Stan Birchfield, Irfan Essa +1
We present an efficient and scalable algorithm for segmenting 3D RGBD point clouds by combining depth, color, and temporal information using a multistage, hierarchical graph-based…
Let's Dance: Learning From Online Dance Videos
Daniel Castro, Steven Hickson, Patsorn Sangkloy +4
In recent years, deep neural network approaches have naturally extended to the video domain, in their simplest case by aggregating per-frame classifications as a baseline for actio…
DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks
Unaiza Ahsan, Chen Sun, Irfan Essa
We propose an action recognition framework using Gen- erative Adversarial Networks. Our model involves train- ing a deep convolutional generative adversarial network (DCGAN) using…
Object category learning and retrieval with weak supervision
Steven Hickson, Anelia Angelova, Irfan Essa +1
We consider the problem of retrieving objects from image data and learning to classify them into meaningful semantic categories with minimal supervision. To that end, we propose a…
Automated Surgical Skill Assessment in RMIS Training
Aneeq Zia, Irfan Essa
Purpose: Manual feedback in basic RMIS training can consume a significant amount of time from expert surgeons' schedule and is prone to subjectivity. While VR-based training tasks…
One-Shot Learning for Semantic Segmentation
Amirreza Shaban, Shray Bansal, Zhen Liu +2
Low-shot learning methods for image classification support learning from sparse data. We extend these techniques to support dense semantic image segmentation. Specifically, we trai…
Semantic Instance Labeling Leveraging Hierarchical Segmentation
Steven Hickson, Irfan Essa, Henrik Christensen
Most of the approaches for indoor RGBD semantic la- beling focus on using pixels or superpixels to train a classi- fier. In this paper, we implement a higher level segmentation usi…
Eyemotion: Classifying facial expressions in VR using eye-tracking cameras
Steven Hickson, Nick Dufour, Avneesh Sud +2
One of the main challenges of social interaction in virtual reality settings is that head-mounted displays occlude a large portion of the face, blocking facial expressions and ther…
Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment
Aneeq Zia, Yachna Sharma, Vinay Bettadapura +2
Purpose: Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save exper…
Complex Event Recognition from Images with Few Training Examples
Unaiza Ahsan, Chen Sun, James Hays +1
We propose to leverage concept-level representations for complex event recognition in photographs given limited training examples. We introduce a novel framework to discover event…
Discovering Picturesque Highlights from Egocentric Vacation Videos
Vinay Bettadapura, Daniel Castro, Irfan Essa
We present an approach for identifying picturesque highlights from large amounts of egocentric video data. Given a set of egocentric videos captured over the course of a vacation,…
Finding Temporally Consistent Occlusion Boundaries in Videos using Geometric Context
S. Hussain Raza, Ahmad Humayun, Matthias Grundmann +2
We present an algorithm for finding temporally consistent occlusion boundaries in videos to support segmentation of dynamic scenes. We learn occlusion boundaries in a pairwise Mark…
Geometric Context from Videos
S. Hussain Raza, Matthias Grundmann, Irfan Essa
We present a novel algorithm for estimating the broad 3D geometric structure of outdoor video scenes. Leveraging spatio-temporal video segmentation, we decompose a dynamic scene ca…
Depth Extraction from Videos Using Geometric Context and Occlusion Boundaries
S. Hussain Raza, Omar Javed, Aveek Das +3
We present an algorithm to estimate depth in dynamic video scenes. We propose to learn and infer depth in videos from appearance, motion, occlusion boundaries, and geometric contex…
Leveraging Context to Support Automated Food Recognition in Restaurants
Vinay Bettadapura, Edison Thomaz, Aman Parnami +2
The pervasiveness of mobile cameras has resulted in a dramatic increase in food photos, which are pictures reflecting what people eat. In this paper, we study how taking pictures o…
Egocentric Field-of-View Localization Using First-Person Point-of-View Devices
Vinay Bettadapura, Irfan Essa, Caroline Pantofaru
We present a technique that uses images, videos and sensor data taken from first-person point-of-view devices to perform egocentric field-of-view (FOV) localization. We define egoc…