37 citations · 147 across the 22 of their papers we have counts for
25 papers
Audio System One Model: Typed Probabilistic Perception of Emotion in Speech
Jaime Bellver-Soler, Iván Martín-Fernández, Anmol Guragain +1
Speech applications ask many questions about the same voice: which emotion a listener perceives, how aroused the speaker sounds, whether they sound frustrated. Classifiers answer f…
Mixture of Phonetic Experts Based Low-Rank Adaptation of Conformer Models for Accented English Speech Recognition
Santosh Dahal, Anmol Guragain, Tianchi Liu +2
We present Mixture of Phonetic Experts based Low-Rank Adaptation (MoPE-LoRA), a parameter-efficient adaptation framework for accented English speech recognition. Recent mixture-of-…
thaulab@EEUCA 2026: Who Said What to Whom? A Targeting-Aware Neural-Symbolic Pipeline for Gaming Toxicity Detection
Anmol Guragain, Marcos Estecha-Garitagoitia, Luis Fernando D'Haro Enríquez +1
This paper describes our system for the EEUCA 2026 Shared Task on toxicity classification in gaming chat. We implement a three-stage pipeline combining an ensemble of two compact t…
Overview of Dialog System Evaluation Track: Dimensionality, Language, Culture and Safety at DSTC 12
John Mendonça, Lining Zhang, Rahul Mallidi +4
The rapid advancement of Large Language Models (LLMs) has intensified the need for robust dialogue system evaluation, yet comprehensive assessment remains challenging. Traditional…
Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models
Marcos Estecha-Garitagoitia, Chen Zhang, Mario Rodríguez-Cantelar +1
This paper provides preliminary results on exploring the task of performing turn-level data augmentation for dialogue system based on different types of commonsense relationships,…
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models
Yiming Chen, Xianghu Yue, Xiaoxue Gao +4
Various audio-LLMs (ALLMs) have been explored recently for tackling different audio tasks simultaneously using a single, unified model. While existing evaluations of ALLMs primaril…
CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo +73
Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge pres…
Unsupervised Mutual Learning of Discourse Parsing and Topic Segmentation in Dialogue
Jiahui Xu, Feng Jiang, Anningzhe Gao +2
In dialogue systems, discourse plays a crucial role in managing conversational focus and coordinating interactions. It consists of two key structures: rhetorical structure and topi…
Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models
Yiming Chen, Chen Zhang, Danqing Luo +3
The automatic evaluation of natural language generation (NLG) systems presents a long-lasting challenge. Recent studies have highlighted various neural metrics that align well with…
Awareness in robotics: An early perspective from the viewpoint of the EIC Pathfinder Challenge "Awareness Inside''
Cosimo Della Santina, Carlos Hernandez Corbato, Burak Sisman +28
Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars…
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
Chen Zhang, Luis Fernando D'Haro, Yiming Chen +2
Automatic evaluation is an integral aspect of dialogue system research. The traditional reference-based NLG metrics are generally found to be unsuitable for dialogue assessment. Co…
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark
Chen Zhang, Luis Fernando D'Haro, Chengguang Tang +3
Recent advancements in reference-free learned metrics for open-domain dialogue evaluation have been driven by the progress in pre-trained language models and the availability of di…
Overview of Robust and Multilingual Automatic Evaluation Metrics for Open-Domain Dialogue Systems at DSTC 11 Track 4
Mario Rodríguez-Cantelar, Chen Zhang, Chengguang Tang +5
The advent and fast development of neural networks have revolutionized the research on dialogue systems and subsequently have triggered various challenges regarding their automatic…
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Chatbots are expected to be knowledgeable across multiple domains, e.g. for daily chit-chat, exchange of information, and grounding in emotional situations. To effectively measure…
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation
Chen Zhang, Luis Fernando D'Haro, Qiquan Zhang +2
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment. However, they either perform turn-level evaluation…
Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges
Shikib Mehri, Jinho Choi, Luis Fernando D'Haro +13
This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggeste…
MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation
Chen Zhang, Luis Fernando D'Haro, Thomas Friedrichs +1
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations. To measure…
Automatic Evaluation and Moderation of Open-domain Dialogue Systems
Chen Zhang, João Sedoc, Luis Fernando D'Haro +2
The development of Open-Domain Dialogue Systems (ODS)is a trending topic due to the large number of research challenges, large societal and business impact, and advances in the und…
Investigating the Impact of Pre-trained Language Models on Dialog Evaluation
Chen Zhang, Luis Fernando D'Haro, Yiming Chen +2
Recently, there is a surge of interest in applying pre-trained language models (Pr-LM) in automatic open-domain dialog evaluation. Pr-LMs offer a promising direction for addressing…
DynaEval: Unifying Turn and Dialogue Level Evaluation
Chen Zhang, Yiming Chen, Luis Fernando D'Haro +4
A dialogue is essentially a multi-turn interaction among interlocutors. Effective evaluation metrics should reflect the dynamics of such interaction. Existing automatic metrics are…
Overview of the Ninth Dialog System Technology Challenge: DSTC9
Chulaka Gunasekara, Seokhwan Kim, Luis Fernando D'Haro +36
This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in…
Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding
Jiewen Wu, Luis Fernando D'Haro, Nancy F. Chen +2
We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding. The proposed approach encodes labels using a combination of…
Dialog System Technology Challenge 7
Koichiro Yoshino, Chiori Hori, Julien Perez +14
This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems. Recently, end-to-end dial…
End-to-End Video Classification with Knowledge Graphs
Fang Yuan, Zhe Wang, Jie Lin +4
Video understanding has attracted much research attention especially since the recent availability of large-scale video benchmarks. In this paper, we address the problem of multi-l…