26 citations · 53 across the 22 of their papers we have counts for
23 papers
MistyPilot: Enabling Social-Robot Control through Multi-Agent LLM Skill Orchestration
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Programming small social robots from natural-language instructions requires more than invoking isolated APIs. Interactive tasks combine reactive physical behaviors with stateful so…
ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videos
Lu Dong, Xiao Wang, Mark Frank +3
Recognizing and localizing student confusion from video is an important yet challenging problem in educational AI. Existing confusion datasets suffer from noisy labels, coarse temp…
InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading
Xiao Wang, Lu Dong, Ifeoma Nwogu +2
Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregive…
MistyPilot: An Agentic Fast-Slow Thinking LLM Framework for Misty Social Robots
Xiao Wang, Lu Dong, Jingchen Sun +3
With the availability of open APIs in social robots, it has become easier to customize general-purpose tools to meet users' needs. However, interpreting high-level user instruction…
MCAD: Multimodal Context-Aware Audio Description Generation For Soccer
Lipisha Chaudhary, Trisha Mittal, Subhadra Gopalakrishnan +2
Audio Descriptions (AD) are essential for making visual content accessible to individuals with visual impairments. Recent works have shown a promising step towards automating AD, b…
AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot
Xiao Wang, Lu Dong, Sahana Rangasrinivasan +3
The social robot's open API allows users to customize open-domain interactions. However, it remains inaccessible to those without programming experience. In this work, we introduce…
Towards the Synthesis of Non-speech Vocalizations
Enjamamul Hoq, Ifeoma Nwogu
In this report, we focus on the unconditional generation of infant cry sounds using the DiffWave framework, which has shown great promise in generating high-quality audio from nois…
Cross-Attention Based Influence Model for Manual and Nonmanual Sign Language Analysis
Lipisha Chaudhary, Fei Xu, Ifeoma Nwogu
Both manual (relating to the use of hands) and non-manual markers (NMM), such as facial expressions or mouthing cues, are important for providing the complete meaning of phrases in…
Ig3D: Integrating 3D Face Representations in Facial Expression Inference
Lu Dong, Xiao Wang, Srirangaraj Setlur +2
Reconstructing 3D faces with facial geometry from single images has allowed for major advances in animation, generative models, and virtual reality. However, this ability to repres…
Towards Open Domain Text-Driven Synthesis of Multi-Person Motions
Mengyi Shan, Lu Dong, Yutao Han +5
This work aims to generate natural and diverse group motions of multiple humans from textual descriptions. While single-person text-to-motion generation is extensively studied, it…
SignAvatar: Sign Language 3D Motion Reconstruction and Generation
Lu Dong, Lipisha Chaudhary, Fei Xu +3
Achieving expressive 3D motion reconstruction and automatic generation for isolated sign words can be challenging, due to the lack of real-world 3D sign-word data, the complex nuan…
Language-guided Human Motion Synthesis with Atomic Actions
Yuanhao Zhai, Mingzhen Huang, Tianyu Luan +5
Language-guided human motion synthesis has been a challenging task due to the inherent complexity and diversity of human behaviors. Previous methods face limitations in generalizat…
SignNet: Single Channel Sign Generation using Metric Embedded Learning
Tejaswini Ananthanarayana, Lipisha Chaudhary, Ifeoma Nwogu
A true interpreting agent not only understands sign language and translates to text, but also understands text and translates to signs. Much of the AI work in sign language transla…
A Probabilistic Model Of Interaction Dynamics for Dyadic Face-to-Face Settings
Renke Wang, Ifeoma Nwogu
Natural conversations between humans often involve a large number of non-verbal nuanced expressions, displayed at key times throughout the conversation. Understanding and being abl…
A Robust Backpropagation-Free Framework for Images
Timothy Zee, Alexander G. Ororbia, Ankur Mali +1
While current deep learning algorithms have been successful for a wide variety of artificial intelligence (AI) tasks, including those involving structured image data, they present…
WLV-RIT at HASOC-Dravidian-CodeMix-FIRE2020: Offensive Language Identification in Code-switched YouTube Comments
Tharindu Ranasinghe, Sarthak Gupte, Marcos Zampieri +1
This paper describes the WLV-RIT entry to the Hate Speech and Offensive Content Identification in Indo-European Languages (HASOC) shared task 2020. The HASOC 2020 organizers provid…
Modeling Global Body Configurations in American Sign Language
Nicholas Wilkins, Beck Cordes Galbraith, Ifeoma Nwogu
American Sign Language (ASL) is the fourth most commonly used language in the United States and is the language most commonly used by Deaf people in the United States and the Engli…
Regression with Uncertainty Quantification in Large Scale Complex Data
Nicholas Wilkins, Michael Johnson, Ifeoma Nwogu
While several methods for predicting uncertainty on deep networks have been recently proposed, they do not readily translate to large and complex datasets. In this paper we utilize…
Computational Social Dynamics: Analyzing the Face-level Interactions in a Group
Nicholas Watkins, Ifeoma Nwogu
Interactional synchrony refers to how the speech or behavior of two or more people involved in a conversation become more finely synchronized with each other, and they can appear t…
Dimensionality Reduction with Subspace Structure Preservation
Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju
Modeling data as being sampled from a union of independent subspaces has been widely applied to a number of real world applications. However, dimensionality reduction approaches th…
A Concept Learning Approach to Multisensory Object Perception
Ifeoma Nwogu, Goker Erdogan, Ilker Yildirim +1
This paper presents a computational model of concept learning using Bayesian inference for a grammatically structured hypothesis space, and test the model on multisensory (visual a…
Is Joint Training Better for Deep Auto-Encoders?
Yingbo Zhou, Devansh Arpit, Ifeoma Nwogu +1
Traditionally, when generative models of data are developed via deep architectures, greedy layer-wise pre-training is employed. In a well-trained model, the lower layer of the arch…
An Analysis of Random Projections in Cancelable Biometrics
Devansh Arpit, Ifeoma Nwogu, Gaurav Srivastava +1
With increasing concerns about security, the need for highly secure physical biometrics-based authentication systems utilizing \emph{cancelable biometric} technologies is on the ri…