44 citations · 90 across the 34 of their papers we have counts for
39 papers
Sense it with your eyes: Sensation Generation and Understanding for Advertisements
Aysan Aghazadeh, Sina Malakouti, Adriana Kovashka
Sensory advertising evokes human senses through visual cues, enabling audiences to mentally simulate experiences and increasing persuasive impact. Despite the recent increase in us…
Learning Consistent Temporal Grounding between Related Tasks in Sports Coaching
Arushi Rai, Adriana Kovashka
Video-LLMs often attend to irrelevant frames, which is especially detrimental for sports coaching tasks requiring precise temporal grounding. Yet obtaining frame-level supervision…
Generalizing Sports Feedback Generation by Watching Competitions and Reading Books: A Rock Climbing Case Study
Arushi Rai, Adriana Kovashka
While there is rapid progress in video-LLMs with advanced reasoning capabilities, prior work shows that these models struggle on the challenging task of sports feedback generation…
Culture in Action: Evaluating Text-to-Image Models through Social Activities
Sina Malakouti, Boqing Gong, Adriana Kovashka
Text-to-image (T2I) diffusion models achieve impressive photorealism by training on large-scale web data, but models inherit cultural biases and fail to depict underrepresented reg…
The Face of Persuasion: Analyzing Bias and Generating Culture-Aware Ads
Aysan Aghazadeh, Adriana Kovashka
Text-to-image models are appealing for customizing visual advertisements and targeting specific populations. We investigate this potential by examining the demographic bias within…
A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modeling
Kyle Buettner, Jacob T. Emmerson, Adriana Kovashka
When captioning an image, people describe objects in diverse ways, such as by using different terms and/or including details that are perceptually noteworthy to them. Descriptions…
Towards Generalization of Tactile Image Generation: Reference-Free Evaluation in a Leakage-Free Setting
Cagri Gungor, Derek Eppinger, Adriana Kovashka
Tactile sensing, which relies on direct physical contact, is critical for human perception and underpins applications in computer vision, robotics, and multimodal learning. Because…
Role Bias in Diffusion Models: Diagnosing and Mitigating through Intermediate Decomposition
Sina Malakouti, Adriana Kovashka
Text-to-image (T2I) diffusion models exhibit impressive photorealistic image generation capabilities, yet they struggle in compositional image generation. In this work, we introduc…
Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity
Zhaoyi Joey Hou, Adriana Kovashka, Xiang Lorraine Li
Evaluating creativity is challenging, even for humans, not only because of its subjectivity but also because it involves complex cognitive processes. Inspired by work in marketing,…
CAP: Evaluation of Persuasive and Creative Image Generation
Aysan Aghazadeh, Adriana Kovashka
We address the task of advertisement image generation and introduce three evaluation metrics to assess Creativity, prompt Alignment, and Persuasiveness (CAP) in generated advertise…
Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval
Kyle Buettner, Adriana Kovashka
There is a scarcity of multilingual vision-language models that properly account for the perceptual differences that are reflected in image captions across languages and cultures.…
Enhancing Weakly-Supervised Object Detection on Static Images through (Hallucinated) Motion
Cagri Gungor, Adriana Kovashka
While motion has garnered attention in various tasks, its potential as a modality for weakly-supervised object detection (WSOD) in static images remains unexplored. Our study intro…
Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognition
Cagri Gungor, Adriana Kovashka
First-person activity recognition is rapidly growing due to the widespread use of wearable cameras but faces challenges from domain shifts across different environments, such as va…
Benchmarking VLMs' Reasoning About Persuasive Atypical Images
Sina Malakouti, Aysan Aghazadeh, Ashmit Khandelwal +1
Vision language models (VLMs) have shown strong zero-shot generalization across various tasks, especially when integrated with large language models (LLMs). However, their ability…
Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition
Kyle Buettner, Sina Malakouti, Xiang Lorraine Li +1
Existing object recognition models have been shown to lack robustness in diverse geographical scenarios due to domain shifts in design and context. Class representations need to be…
Semi-Supervised Domain Generalization for Object Detection via Language-Guided Feature Alignment
Sina Malakouti, Adriana Kovashka
Existing domain adaptation (DA) and generalization (DG) methods in object detection enforce feature alignment in the visual space but face challenges like object appearance variabi…
Hypernymization of named entity-rich captions for grounding-based multi-modal pretraining
Giacomo Nebbia, Adriana Kovashka
Named entities are ubiquitous in text that naturally accompanies images, especially in domains such as news or Wikipedia articles. In previous work, named entities have been identi…
Boosting Weakly Supervised Object Detection using Fusion and Priors from Hallucinated Depth
Cagri Gungor, Adriana Kovashka
Despite recent attention and exploration of depth for various tasks, it is still an unexplored modality for weakly-supervised object detection (WSOD). We propose an amplifier metho…
Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detection
Kyle Buettner, Adriana Kovashka
Vision-language alignment learned from image-caption pairs has been shown to benefit tasks like object recognition and detection. Methods are mostly evaluated in terms of how well…
VEIL: Vetting Extracted Image Labels from In-the-Wild Captions for Weakly-Supervised Object Detection
Arushi Rai, Adriana Kovashka
The use of large-scale vision-language datasets is limited for object detection due to the negative impact of label noise on localization. Prior methods have shown how such large-s…
Contrastive View Design Strategies to Enhance Robustness to Domain Shifts in Downstream Object Detection
Kyle Buettner, Adriana Kovashka
Contrastive learning has emerged as a competitive pretraining method for object detection. Despite this progress, there has been minimal investigation into the robustness of contra…
Weakly-Supervised Action Detection Guided by Audio Narration
Keren Ye, Adriana Kovashka
Videos are more well-organized curated data sources for visual concept learning than images. Unlike the 2-dimensional images which only involve the spatial information, the additio…
Linguistic Structures as Weak Supervision for Visual Scene Graph Generation
Keren Ye, Adriana Kovashka
Prior work in scene graph generation requires categorical supervision at the level of triplets - subjects and objects, and predicates that relate them, either with or without bound…
BasisNet: Two-stage Model Synthesis for Efficient Inference
Mingda Zhang, Chun-Te Chu, Andrey Zhmoginov +6
In this work, we present BasisNet which combines recent advancements in efficient neural network architectures, conditional computation, and early termination in a simple new form.…