1 citations · 1 across the 9 of their papers we have counts for
10 papers
From Generative to Discriminative: Lightweight Adaptation of SpeechLLMs for Multimodal Emotion Recognition
Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello +4
SpeechLLMs can recognise emotion by generating a label token, but this approach relies on the LLM's existing output representations. We compare this generative approach with a disc…
When Synthetic Speech Is All You Have: Better Call GRPO
Shashi Kumar, Yanis Labrak, Hasindri Watawana +5
LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TT…
Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR
Thibault Bañeras-Roux, Sergio Burdisso, Esaú Villatoro-Tello +9
Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR architectures connect a speech encode…
Abstract Compression of Conversational Audio Context for LLM-based ASR
Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso +7
Standard LLM-based speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether multimodal context f…
When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews
Hasindri Watawana, Sergio Burdisso, Diego A. Moreno-Galván +4
Automatic depression detection from doctor-patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interp…
Uncertainty Awareness Enables Efficient Labeling for Cancer Subtyping in Digital Pathology
Nirhoshan Sivaroopan, Chamuditha Jayanga Galappaththige, Chalani Ekanayake +4
Machine-learning-assisted cancer subtyping is a promising avenue in digital pathology. Cancer subtyping models, however, require careful training using expert annotations so that t…
A Culturally-diverse Multilingual Multimodal Video Benchmark & Model
Bhuiyan Sanjid Shafique, Ashmal Vayani, Muhammad Maaz +26
Large multimodal models (LMMs) have recently gained attention due to their effectiveness to understand and generate descriptions of visual content. Most existing LMMs are in Englis…
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana +66
Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cul…
Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
Hasindri Watawana, Kanchana Ranasinghe, Tariq Mahmood +3
Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn…
Contrastive Deep Encoding Enables Uncertainty-aware Machine-learning-assisted Histopathology
Nirhoshan Sivaroopan, Chamuditha Jayanga, Chalani Ekanayake +6
Deep neural network models can learn clinically relevant features from millions of histopathology images. However generating high-quality annotations to train such models for each…