24 citations · 83 across the 67 of their papers we have counts for
13 papers · 1 filter
MAGIC-Talk: Motion-aware Audio-Driven Talking Face Generation with Customizable Identity Control
Fatemeh Nazarieh, Zhenhua Feng, Diptesh Kanojia +2
Audio-driven talking face generation has gained significant attention for applications in digital media and virtual avatars. While recent methods improve audio-lip synchronization,…
Evaluating Open-Source Vision-Language Models for Multimodal Sarcasm Detection
Saroj Basnet, Shafkat Farabi, Tharindu Ranasinghe +2
Recent advances in open-source vision-language models (VLMs) offer new opportunities for understanding complex and subjective multimodal phenomena such as sarcasm. In this work, we…
The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation
Girish A. Koushik, Fatemeh Nazarieh, Katherine Birch +2
Visual metaphor generation is a challenging task that aims to generate an image given an input text metaphor. Inherently, it needs language understanding to bind a source concept w…
Cyberbullying Detection via Aggression-Enhanced Prompting
Aisha Saeid, Anu Sabu, Girish A. Koushik +2
Detecting cyberbullying on social media remains a critical challenge due to its subtle and varied expressions. This study investigates whether integrating aggression detection as a…
ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models
Archchana Sindhujan, Shenbin Qian, Chan Chi Chun Matthew +2
Large Language Models (LLMs) have shown remarkable performance across a wide range of natural language processing tasks. Quality Estimation (QE) for Machine Translation (MT), which…
NEAR: A Nested Embedding Approach to Efficient Product Retrieval and Ranking
Shenbin Qian, Diptesh Kanojia, Samarth Agrawal +4
E-commerce information retrieval (IR) systems struggle to simultaneously achieve high accuracy in interpreting complex user queries and maintain efficient processing of vast produc…
TRACE: Textual Relevance Augmentation and Contextual Encoding for Multimodal Hate Detection
Girish A. Koushik, Helen Treharne, Aditya Joshi +1
Social media memes are a challenging domain for hate detection because they intertwine visual and textual cues into culturally nuanced messages. To tackle these challenges, we intr…
Automatically Generating Chinese Homophone Words to Probe Machine Translation Estimation Systems
Shenbin Qian, Constantin Orăsan, Diptesh Kanojia +1
Evaluating machine translation (MT) of user-generated content (UGC) involves unique challenges such as checking whether the nuance of emotions from the source are preserved in the…
DGFM: Full Body Dance Generation Driven by Music Foundation Models
Xinran Liu, Zhenhua Feng, Diptesh Kanojia +1
In music-driven dance motion generation, most existing methods use hand-crafted features and neglect that music foundation models have profoundly impacted cross-modal content gener…
Towards a Robust Framework for Multimodal Hate Detection: A Study on Video vs. Image-based Content
Girish A. Koushik, Diptesh Kanojia, Helen Treharne
Social media platforms enable the propagation of hateful content across different modalities such as textual, auditory, and visual, necessitating effective detection methods. While…
GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation
Xinran Liu, Xu Dong, Shenbin Qian +3
Music-driven dance generation is a challenging task as it requires strict adherence to genre-specific choreography while ensuring physically realistic and precisely synchronized da…
Giving the Old a Fresh Spin: Quality Estimation-Assisted Constrained Decoding for Automatic Post-Editing
Sourabh Deoghare, Diptesh Kanojia, Pushpak Bhattacharyya
Automatic Post-Editing (APE) systems often struggle with over-correction, where unnecessary modifications are made to a translation, diverging from the principle of minimal editing…
When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languages
Archchana Sindhujan, Diptesh Kanojia, Constantin Orasan +1
This paper investigates the reference-less evaluation of machine translation for low-resource language pairs, known as quality estimation (QE). Segment-level QE is a challenging cr…