28 citations · 79 across the 28 of their papers we have counts for
23 papers · 1 filter
Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruning
Wenda Qin, Andrea Burns, Bryan A. Plummer +1
Large models achieve strong performance on Vision-and-Language Navigation (VLN) tasks, but are costly to run in resource-limited environments. Token pruning offers appealing tradeo…
Gen-AFFECT: Generation of Avatar Fine-grained Facial Expressions with Consistent identiTy
Hao Yu, Rupayan Mallick, Margrit Betke +1
Different forms of customized 2D avatars are widely used in gaming applications, virtual communication, education, and content creation. However, existing approaches often fail to…
LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning
Kaihong Wang, Donghyun Kim, Margrit Betke
Continual learning for vision-language models has achieved remarkable performance through synthetic replay, where samples are generated using Stable Diffusion to regularize during…
GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces
Hao Yu, Rupayan Mallick, Margrit Betke +1
Cartoon avatars have been widely used in various applications, including social media, online tutoring, and gaming. However, existing cartoon avatar datasets and generation methods…
DebiasPI: Inference-time Debiasing by Prompt Iteration of a Text-to-Image Generative Model
Sarah Bonna, Yu-Cheng Huang, Ekaterina Novozhilova +11
Ethical intervention prompting has emerged as a tool to counter demographic biases of text-to-image generative AI models. Existing solutions either require to retrain the model or…
ExeChecker: Where Did I Go Wrong?
Yiwen Gu, Mahir Patel, Margrit Betke
In this paper, we present a contrastive learning based framework, ExeChecker, for the interpretation of rehabilitation exercises. Our work builds upon state-of-the-art advances in…
BU-CVKit: Extendable Computer Vision Framework for Species Independent Tracking and Analysis
Mahir Patel, Lucas Carstensen, Yiwen Gu +2
A major bottleneck of interdisciplinary computer vision (CV) research is the lack of a framework that eases the reuse and abstraction of state-of-the-art CV models by CV and non-CV…
Exploring Consistency in Cross-Domain Transformer for Domain Adaptive Semantic Segmentation
Kaihong Wang, Donghyun Kim, Rogerio Feris +2
While transformers have greatly boosted performance in semantic segmentation, domain adaptive transformers are not yet well explored. We identify that the domain gap can cause disc…
A graph-transformer for whole slide image classification
Yi Zheng, Rushin H. Gindra, Emily J. Green +4
Deep learning is a powerful tool for whole slide image (WSI) analysis. Typically, when performing supervised deep learning, a WSI is divided into small patches, trained and the out…
A Unified Framework for Domain Adaptive Pose Estimation
Donghyun Kim, Kaihong Wang, Kate Saenko +2
While pose estimation is an important computer vision task, it requires expensive annotation and suffers from domain shift. In this paper, we investigate the problem of domain adap…
Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
Kaihong Wang, Chenhongyi Yang, Margrit Betke
Unsupervised domain adaptation for semantic segmentation has been intensively studied due to the low cost of the pixel-level annotation for synthetic data. The most common approach…
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Mona Jalal, Josef Spjut, Ben Boudaoud +1
We present a new, publicly-available image dataset generated by the NVIDIA Deep Learning Data Synthesizer intended for use in object detection, pose estimation, and tracking applic…
Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System
Nataniel Ruiz, Hao Yu, Danielle A. Allessio +10
In this work, we propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a stud…
Learning to Separate: Detecting Heavily-Occluded Objects in Urban Scenes
Chenhongyi Yang, Vitaly Ablavsky, Kaihong Wang +2
While visual object detection with deep learning has received much attention in the past decade, cases when heavy intra-class occlusions occur have not been studied thoroughly. In…
A method for detecting text of arbitrary shapes in natural scenes that improves text spotting
Qitong Wang, Yi Zheng, Margrit Betke
Understanding the meaning of text in images of natural scenes like highway signs or store front emblems is particularly challenging if the text is foreshortened in the image or the…
Scraping Social Media Photos Posted in Kenya and Elsewhere to Detect and Analyze Food Types
Kaihong Wang, Mona Jalal, Sankara Jefferson +3
Monitoring population-level changes in diet could be useful for education and for implementing interventions to improve health. Research has shown that data from social media sourc…
Deep Neural Network for Semantic-based Text Recognition in Images
Yi Zheng, Qitong Wang, Margrit Betke
State-of-the-art text spotting systems typically aim to detect isolated words or word-by-word text in images of natural scenes and ignore the semantic coherence within a region of…
Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch
Danna Gurari, Yinan Zhao, Suyog Dutt Jain +2
Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need o…
SAVOIAS: A Diverse, Multi-Category Visual Complexity Dataset
Elham Saraee, Mona Jalal, Margrit Betke
Visual complexity identifies the level of intricacy and details in an image or the level of difficulty to describe the image. It is an important concept in a variety of areas such…
Predicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation(s)
Danna Gurari, Kun He, Bo Xiong +6
We propose the ambiguity problem for the foreground object segmentation task and motivate the importance of estimating and accounting for this ambiguity when designing vision syste…
Automating Image Analysis by Annotating Landmarks with Deep Neural Networks
Mikhail Breslav, Tyson L. Hedrick, Stan Sclaroff +1
Image and video analysis is often a crucial step in the study of animal behavior and kinematics. Often these analyses require that the position of one or more animal landmarks are…
Salient Object Subitizing
Jianming Zhang, Shugao Ma, Mehrnoosh Sameki +6
We study the problem of Salient Object Subitizing, i.e. predicting the existence and the number of salient objects in an image using holistic cues. This task is inspired by the abi…
Discovering Useful Parts for Pose Estimation in Sparsely Annotated Datasets
Mikhail Breslav, Tyson L. Hedrick, Stan Sclaroff +1
Our work introduces a novel way to increase pose estimation accuracy by discovering parts from unannotated regions of training images. Discovered parts are used to generate more ac…