1 citations · 3 across the 16 of their papers we have counts for
19 papers
TRACE: Training-aware Robustness and Challenge Evaluation for Text-Vision Embedding Models
Pume Tuchinda, Tinnakit Udsa, Peerat Limkonchotiwat
Existing benchmarks for text-vision embedding models report aggregate scores that obscure whether performance comes from general capabilities, task-specific adaptation, or robustne…
Validating Causal Abstraction Metrics on Simulated Complex Systems
Maxime Méloux, Tiago Pimentel, François Portet +1
A central goal of science is to produce valid explanations of complex systems: high-level causal accounts that faithfully reflect the behavior of lower-level mechanisms. Yet no con…
Speaker Group Encoding in Self-supervised Speech Recognition Models
Felix Herron, Solange Rossato Alexandre Allauzen, Benoit Favre +1
We investigate what self-supervised speech recognition models (S3Ms) learn about speaker groups (SGs). We examine several states of S3Ms: pretrained, finetuned on speaker identific…
Responsible Benchmarking of Fairness for Automatic Speech Recognition
Felix Herron, Ange Richard, François Portet +2
Many studies have shown automatic speech processing (ASR) systems have unequal performance across speakergroups (SG's). However, the manner in which such studies arrive at this con…
MedMeta: A Benchmark for LLMs in Synthesizing Meta-Analysis Conclusion from Medical Studies
Huy Hoang Ha, Benoit Favre, Francois Portet
Large language models (LLMs) have saturated standard medical benchmarks that test factual recall, yet their ability to perform higher-order reasoning, such as synthesizing evidence…
Automated Clinical Report Generation for Remote Cognitive Remediation: Comparing Knowledge-Engineered Templates and LLMs in Low-Resource Settings
Yongxin Zhou, Fabien Ringeval, François Portet
The growing demand for cognitive remediation therapy, combined with limited speech therapist availability, has accelerated the adoption of remote rehabilitation tools. These system…
Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition models
Felix Herron, Solange Rossato, Alexandre Allauzen +1
Modern automatic speech recognition (ASR) systems have been observed to function better for certain speaker groups (SGs) than others, despite recent gains in overall performance. O…
Where Do Self-Supervised Speech Models Become Unfair?
Felix Herron, Maja Hjuler, Solange Rossato +2
Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing why this occurs on a technolog…
The Dead Salmons of AI Interpretability
Maxime Méloux, Giada Dirupo, François Portet +1
In a striking neuroscience study, the authors placed a dead salmon in an MRI scanner and showed it images of humans in social situations. Astonishingly, standard analyses of the ti…
Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
Maxime Méloux, François Portet, Maxime Peyrard
Mechanistic Interpretability (MI) aims to reverse-engineer model behaviors by identifying functional sub-networks. Yet, the scientific validity of these findings depends on their s…
Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
Maxime Méloux, Silviu Maniu, François Portet +1
As AI systems are used in high-stakes applications, ensuring interpretability is crucial. Mechanistic Interpretability (MI) aims to reverse-engineer neural networks by extracting h…
THERADIA WoZ: An Ecological Corpus for Appraisal-based Affect Research in Healthcare
Hippolyte Fournier, Sina Alisamir, Safaa Azzakhnini +14
We present THERADIA WoZ, an ecological corpus designed for audiovisual research on affect in healthcare. Two groups of senior individuals, consisting of 52 healthy participants and…
Spoken Dialogue System for Medical Prescription Acquisition on Smartphone: Development, Corpus and Evaluation
Ali Can Kocabiyikoglu, François Portet, Jean-Marc Babouchkine +3
Hospital information systems (HIS) have become an essential part of healthcare institutions and now incorporate prescribing support software. Prescription support software allows f…
Can GPT models Follow Human Summarization Guidelines? A Study for Targeted Communication Goals
Yongxin Zhou, Fabien Ringeval, François Portet
This study investigates the ability of GPT models (ChatGPT, GPT-4 and GPT-4o) to generate dialogue summaries that adhere to human guidelines. Our evaluation involved experimenting…
FRACAS: A FRench Annotated Corpus of Attribution relations in newS
Ange Richard, Laura Alonzo-Canul, François Portet
Quotation extraction is a widely useful task both from a sociological and from a Natural Language Processing perspective. However, very little data is available to study this task…
PSentScore: Evaluating Sentiment Polarity in Dialogue Summarization
Yongxin Zhou, Fabien Ringeval, François Portet
Automatic dialogue summarization is a well-established task with the goal of distilling the most crucial information from human conversations into concise textual summaries. Howeve…
Effectiveness of French Language Models on Abstractive Dialogue Summarization Task
Yongxin Zhou, François Portet, Fabien Ringeval
Pre-trained language models have established the state-of-the-art on various natural language processing tasks, including dialogue summarization, which allows the reader to quickly…
A Spoken Drug Prescription Dataset in French for Spoken Language Understanding
Ali Can Kocabiyikoglu, François Portet, Prudence Gibert +3
Spoken medical dialogue systems are increasingly attracting interest to enhance access to healthcare services and improve quality and traceability of patient care. In this paper, w…
GenderedNews: Une approche computationnelle des écarts de représentation des genres dans la presse française
Ange Richard, Gilles Bastin, François Portet
In this article, we present {\it GenderedNews} (\url{https://gendered-news.imag.fr}), an online dashboard which gives weekly measures of gender imbalance in French online press. We…