28 citations · 79 across the 28 of their papers we have counts for
32 papers
Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruning
Wenda Qin, Andrea Burns, Bryan A. Plummer +1
Large models achieve strong performance on Vision-and-Language Navigation (VLN) tasks, but are costly to run in resource-limited environments. Token pruning offers appealing tradeo…
Gen-AFFECT: Generation of Avatar Fine-grained Facial Expressions with Consistent identiTy
Hao Yu, Rupayan Mallick, Margrit Betke +1
Different forms of customized 2D avatars are widely used in gaming applications, virtual communication, education, and content creation. However, existing approaches often fail to…
LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning
Kaihong Wang, Donghyun Kim, Margrit Betke
Continual learning for vision-language models has achieved remarkable performance through synthetic replay, where samples are generated using Stable Diffusion to regularize during…
GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces
Hao Yu, Rupayan Mallick, Margrit Betke +1
Cartoon avatars have been widely used in various applications, including social media, online tutoring, and gaming. However, existing cartoon avatar datasets and generation methods…
DebiasPI: Inference-time Debiasing by Prompt Iteration of a Text-to-Image Generative Model
Sarah Bonna, Yu-Cheng Huang, Ekaterina Novozhilova +11
Ethical intervention prompting has emerged as a tool to counter demographic biases of text-to-image generative AI models. Existing solutions either require to retrain the model or…
ExeChecker: Where Did I Go Wrong?
Yiwen Gu, Mahir Patel, Margrit Betke
In this paper, we present a contrastive learning based framework, ExeChecker, for the interpretation of rehabilitation exercises. Our work builds upon state-of-the-art advances in…
A Lesion-aware Edge-based Graph Neural Network for Predicting Language Ability in Patients with Post-stroke Aphasia
Zijian Chen, Maria Varkanitsa, Prakash Ishwar +4
We propose a lesion-aware graph neural network (LEGNet) to predict language ability from resting-state fMRI (rs-fMRI) connectivity in patients with post-stroke aphasia. Our model i…
Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation
Ge Gao, Jongin Kim, Sejin Paik +5
Predicting emotions elicited by news headlines can be challenging as the task is largely influenced by the varying nature of people's interpretations and backgrounds. Previous work…
Detecting Frames in News Headlines and Lead Images in U.S. Gun Violence Coverage
Isidora Chara Tourni, Lei Guo, Hengchang Hu +9
News media structure their reporting of events or issues using certain perspectives. When describing an incident involving gun violence, for example, some journalists may focus on…
BU-CVKit: Extendable Computer Vision Framework for Species Independent Tracking and Analysis
Mahir Patel, Lucas Carstensen, Yiwen Gu +2
A major bottleneck of interdisciplinary computer vision (CV) research is the lack of a framework that eases the reuse and abstraction of state-of-the-art CV models by CV and non-CV…
Exploring Consistency in Cross-Domain Transformer for Domain Adaptive Semantic Segmentation
Kaihong Wang, Donghyun Kim, Rogerio Feris +2
While transformers have greatly boosted performance in semantic segmentation, domain adaptive transformers are not yet well explored. We identify that the domain gap can cause disc…
A graph-transformer for whole slide image classification
Yi Zheng, Rushin H. Gindra, Emily J. Green +4
Deep learning is a powerful tool for whole slide image (WSI) analysis. Typically, when performing supervised deep learning, a WSI is divided into small patches, trained and the out…
A Unified Framework for Domain Adaptive Pose Estimation
Donghyun Kim, Kaihong Wang, Kate Saenko +2
While pose estimation is an important computer vision task, it requires expensive annotation and suffers from domain shift. In this paper, we investigate the problem of domain adap…
Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
Kaihong Wang, Chenhongyi Yang, Margrit Betke
Unsupervised domain adaptation for semantic segmentation has been intensively studied due to the low cost of the pixel-level annotation for synthetic data. The most common approach…
OpenFraming: We brought the ML; you bring the data. Interact with your data and discover its frames
Alyssa Smith, David Assefa Tofu, Mona Jalal +7
When journalists cover a news story, they can cover the story from multiple angles or perspectives. A news article written about COVID-19 for example, might focus on personal preve…
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Mona Jalal, Josef Spjut, Ben Boudaoud +1
We present a new, publicly-available image dataset generated by the NVIDIA Deep Learning Data Synthesizer intended for use in object detection, pose estimation, and tracking applic…
Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System
Nataniel Ruiz, Hao Yu, Danielle A. Allessio +10
In this work, we propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a stud…
Performance Comparison of Crowdworkers and NLP Tools on Named-Entity Recognition and Sentiment Analysis of Political Tweets
Mona Jalal, Kate K. Mays, Lei Guo +1
We report results of a comparison of the accuracy of crowdworkers and seven Natural Language Processing (NLP) toolkits in solving two important NLP tasks, named-entity recognition…
Learning to Separate: Detecting Heavily-Occluded Objects in Urban Scenes
Chenhongyi Yang, Vitaly Ablavsky, Kaihong Wang +2
While visual object detection with deep learning has received much attention in the past decade, cases when heavy intra-class occlusions occur have not been studied thoroughly. In…
A method for detecting text of arbitrary shapes in natural scenes that improves text spotting
Qitong Wang, Yi Zheng, Margrit Betke
Understanding the meaning of text in images of natural scenes like highway signs or store front emblems is particularly challenging if the text is foreshortened in the image or the…
Scraping Social Media Photos Posted in Kenya and Elsewhere to Detect and Analyze Food Types
Kaihong Wang, Mona Jalal, Sankara Jefferson +3
Monitoring population-level changes in diet could be useful for education and for implementing interventions to improve health. Research has shown that data from social media sourc…
Deep Neural Network for Semantic-based Text Recognition in Images
Yi Zheng, Qitong Wang, Margrit Betke
State-of-the-art text spotting systems typically aim to detect isolated words or word-by-word text in images of natural scenes and ignore the semantic coherence within a region of…
Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch
Danna Gurari, Yinan Zhao, Suyog Dutt Jain +2
Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need o…
BUOCA: Budget-Optimized Crowd Worker Allocation
Mehrnoosh Sameki, Sha Lai, Kate K. Mays +3
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resultin…