17 citations · 27 across the 23 of their papers we have counts for
24 papers
Tellimation: Making Narrative Gaps Visible in Children's Storytelling with Just-in-Time Animation
Vincent Cavez, Marielle Zheng, Momin Siddiqui +4
Illustrated scenes are used to prompt children's stories, yet children with language difficulties often omit the details, actions, and relationships that make a story coherent. We…
MistyPilot: Enabling Social-Robot Control through Multi-Agent LLM Skill Orchestration
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Programming small social robots from natural-language instructions requires more than invoking isolated APIs. Interactive tasks combine reactive physical behaviors with stateful so…
ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videos
Lu Dong, Xiao Wang, Mark Frank +3
Recognizing and localizing student confusion from video is an important yet challenging problem in educational AI. Existing confusion datasets suffer from noisy labels, coarse temp…
InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregive…
MistyPilot: An Agentic Fast-Slow Thinking LLM Framework for Misty Social Robots
Xiao Wang, Lu Dong, Jingchen Sun +3
With the availability of open APIs in social robots, it has become easier to customize general-purpose tools to meet users' needs. However, interpreting high-level user instruction…
LLM Augmented Intervenable Multimodal Adaptor for Post-operative Complication Prediction in Lung Cancer Surgery
Shubham Pandey, Bhavin Jawade, Srirangaraj Setlur +2
Postoperative complications remain a critical concern in clinical practice, adversely affecting patient outcomes and contributing to rising healthcare costs. We present MIRACLE, a…
Teaching Spell Checkers to Teach: Pedagogical Program Synthesis for Interactive Learning
Momin N. Siddiqui, Vincent Cavez, Sahana Rangasrinivasan +4
Spelling taught through memorization often fails many learners, particularly children with language-based learning disorders who struggle with the phonological skills necessary to…
Ridgeformer: Mutli-Stage Contrastive Training For Fine-grained Cross-Domain Fingerprint Recognition
Shubham Pandey, Bhavin Jawade, Srirangaraj Setlur
The increasing demand for hygienic and portable biometric systems has underscored the critical need for advancements in contactless fingerprint recognition. Despite its potential,…
AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot
Xiao Wang, Lu Dong, Sahana Rangasrinivasan +3
The social robot's open API allows users to customize open-domain interactions. However, it remains inaccessible to those without programming experience. In this work, we introduce…
SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval
Bhavin Jawade, Joao V. B. Soares, Kapil Thadani +6
Compositional image retrieval (CIR) is a multimodal learning task where a model combines a query image with a user-provided text modification to retrieve a target image. CIR finds…
RealCQA-V2: A Diagnostic Benchmark for Structured Visual Entailment over Scientific Charts
Saleem Ahmed, Srirangaraj Setlur, Venu Govindaraju
Multimodal reasoning models often produce fluent answers supported by seemingly coherent rationales. Existing benchmarks evaluate only final-answer correctness. They do not support…
Ig3D: Integrating 3D Face Representations in Facial Expression Inference
Lu Dong, Xiao Wang, Srirangaraj Setlur +2
Reconstructing 3D faces with facial geometry from single images has allowed for major advances in animation, generative models, and virtual reality. However, this ability to repres…
Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos
Dennis Fedorishin, Lie Lu, Srirangaraj Setlur +1
A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual…
Fine-Grained Engine Fault Sound Event Detection Using Multimodal Signals
Dennis Fedorishin, Livio Forte, Philip Schneider +2
Sound event detection (SED) is an active area of audio research that aims to detect the temporal occurrence of sounds. In this paper, we apply SED to engine fault detection by intr…
Deep Metric Learning for Computer Vision: A Brief Overview
Deen Dayal Mohan, Bhavin Jawade, Srirangaraj Setlur +1
Objective functions that optimize deep neural networks play a vital role in creating an enhanced feature representation of the input data. Although cross-entropy-based loss formula…
Liveness Detection Competition -- Noncontact-based Fingerprint Algorithms and Systems (LivDet-2023 Noncontact Fingerprint)
Sandip Purnapatra, Humaira Rezaie, Bhavin Jawade +19
Liveness Detection (LivDet) is an international competition series open to academia and industry with the objec-tive to assess and report state-of-the-art in Presentation Attack De…
DIOR: Dataset for Indoor-Outdoor Reidentification -- Long Range 3D/2D Skeleton Gait Collection Pipeline, Semi-Automated Gait Keypoint Labeling and Baseline Evaluation Methods
Yuyang Chen, Praveen Raj Masilamani, Bhavin Jawade +2
In recent times, there is an increased interest in the identification and re-identification of people at long distances, such as from rooftop cameras, UAV cameras, street cams, and…
RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logic
Saleem Ahmed, Bhavin Jawade, Shubham Pandey +2
We present a comprehensive study of chart visual question-answering(QA) task, to address the challenges faced in comprehending and extracting data from chart visualizations within…
SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding
Saleem Ahmed, Pengyu Yan, David Doermann +2
We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to recon…
CoNAN: Conditional Neural Aggregation Network For Unconstrained Face Feature Fusion
Bhavin Jawade, Deen Dayal Mohan, Dennis Fedorishin +2
Face recognition from image sets acquired under unregulated and uncontrolled settings, such as at large distances, low resolutions, varying viewpoints, illumination, pose, and atmo…
RidgeBase: A Cross-Sensor Multi-Finger Contactless Fingerprint Dataset
Bhavin Jawade, Deen Dayal Mohan, Srirangaraj Setlur +2
Contactless fingerprint matching using smartphone cameras can alleviate major challenges of traditional fingerprint systems including hygienic acquisition, portability and presenta…
Hear The Flow: Optical Flow-Based Self-Supervised Visual Sound Source Localization
Dennis Fedorishin, Deen Dayal Mohan, Bhavin Jawade +2
Learning to localize the sound source in videos without explicit annotations is a novel area of audio-visual research. Existing work in this area focuses on creating attention maps…
Article citation study: Context enhanced citation sentiment detection
Vishal Vyas, Kumar Ravi, Vadlamani Ravi +3
Citation sentimet analysis is one of the little studied tasks for scientometric analysis. For citation analysis, we developed eight datasets comprising citation sentences, which ar…
CAN: Composite Appearance Network for Person Tracking and How to Model Errors in a Tracking System
Neeti Narayan, Nishant Sankaran, Srirangaraj Setlur +1
Tracking multiple people across multiple cameras is an open problem. It is typically divided into two tasks: (i) single-camera tracking (SCT) - identify trajectories in the same sc…