17 citations · 19 across the 16 of their papers we have counts for
16 papers
MistyPilot: Enabling Social-Robot Control through Multi-Agent LLM Skill Orchestration
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Programming small social robots from natural-language instructions requires more than invoking isolated APIs. Interactive tasks combine reactive physical behaviors with stateful so…
ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videos
Lu Dong, Xiao Wang, Mark Frank +3
Recognizing and localizing student confusion from video is an important yet challenging problem in educational AI. Existing confusion datasets suffer from noisy labels, coarse temp…
InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregive…
MistyPilot: An Agentic Fast-Slow Thinking LLM Framework for Misty Social Robots
Xiao Wang, Lu Dong, Jingchen Sun +3
With the availability of open APIs in social robots, it has become easier to customize general-purpose tools to meet users' needs. However, interpreting high-level user instruction…
LLM Augmented Intervenable Multimodal Adaptor for Post-operative Complication Prediction in Lung Cancer Surgery
Shubham Pandey, Bhavin Jawade, Srirangaraj Setlur +2
Postoperative complications remain a critical concern in clinical practice, adversely affecting patient outcomes and contributing to rising healthcare costs. We present MIRACLE, a…
AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot
Xiao Wang, Lu Dong, Sahana Rangasrinivasan +3
The social robot's open API allows users to customize open-domain interactions. However, it remains inaccessible to those without programming experience. In this work, we introduce…
SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval
Bhavin Jawade, Joao V. B. Soares, Kapil Thadani +6
Compositional image retrieval (CIR) is a multimodal learning task where a model combines a query image with a user-provided text modification to retrieve a target image. CIR finds…
RealCQA-V2: A Diagnostic Benchmark for Structured Visual Entailment over Scientific Charts
Saleem Ahmed, Srirangaraj Setlur, Venu Govindaraju
Multimodal reasoning models often produce fluent answers supported by seemingly coherent rationales. Existing benchmarks evaluate only final-answer correctness. They do not support…
Ig3D: Integrating 3D Face Representations in Facial Expression Inference
Lu Dong, Xiao Wang, Srirangaraj Setlur +2
Reconstructing 3D faces with facial geometry from single images has allowed for major advances in animation, generative models, and virtual reality. However, this ability to repres…
Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos
Dennis Fedorishin, Lie Lu, Srirangaraj Setlur +1
A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual…
Fine-Grained Engine Fault Sound Event Detection Using Multimodal Signals
Dennis Fedorishin, Livio Forte, Philip Schneider +2
Sound event detection (SED) is an active area of audio research that aims to detect the temporal occurrence of sounds. In this paper, we apply SED to engine fault detection by intr…
Deep Metric Learning for Computer Vision: A Brief Overview
Deen Dayal Mohan, Bhavin Jawade, Srirangaraj Setlur +1
Objective functions that optimize deep neural networks play a vital role in creating an enhanced feature representation of the input data. Although cross-entropy-based loss formula…
RealCQA: Scientific Chart Question Answering as a Test-bed for First-Order Logic
Saleem Ahmed, Bhavin Jawade, Shubham Pandey +2
We present a comprehensive study of chart visual question-answering(QA) task, to address the challenges faced in comprehending and extracting data from chart visualizations within…
SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding
Saleem Ahmed, Pengyu Yan, David Doermann +2
We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to recon…
CoNAN: Conditional Neural Aggregation Network For Unconstrained Face Feature Fusion
Bhavin Jawade, Deen Dayal Mohan, Dennis Fedorishin +2
Face recognition from image sets acquired under unregulated and uncontrolled settings, such as at large distances, low resolutions, varying viewpoints, illumination, pose, and atmo…
RidgeBase: A Cross-Sensor Multi-Finger Contactless Fingerprint Dataset
Bhavin Jawade, Deen Dayal Mohan, Srirangaraj Setlur +2
Contactless fingerprint matching using smartphone cameras can alleviate major challenges of traditional fingerprint systems including hygienic acquisition, portability and presenta…