20 citations · 49 across the 34 of their papers we have counts for
35 papers
Reusing Latent Speech Representations for Query-Conditioned Topic Localization in Transcripts
Steffen Freisinger, Philipp Seeberger, Thomas Ranzenberger +2
Long transcripts are costly inputs for downstream NLP systems and often contain irrelevant context. We study query-conditioned topic localization: predicting the sentence span in a…
The Spoken Wikipedia Presentation Corpus
Thomas Ranzenberger, Steffen Freisinger, Tobias Bocklet +1
We present the Spoken Wikipedia Presentation Corpus, an extension of the Spoken Wikipedia Corpora featuring LLM-generated slide decks for multimodal ASR. Slides are created from LL…
Evaluation Pitfalls and Challenges in Multimedia Event Extraction
Philipp Seeberger, Steffen Freisinger, Tobias Bocklet +1
Multimedia event extraction aims to jointly identify events and their arguments across multiple modalities, such as text and images, to support more comprehensive event understandi…
Mitigating Scoring Errors and Compensating for Nonverbal Subtests in Speech-Based Dementia Assessment
Franziska Braun, Christopher Witzl, Andreas Erzigkeit +4
Early detection of cognitive impairment relies on neuropsychological tests to minimize subjectivity by assessing multiple cognitive domains. Speech-based evaluation can support dia…
Time vs. Layer: Locating Predictive Cues for Dysarthric Speech Descriptors in wav2vec 2.0
Natalie Engert, Dominik Wagner, Korbinian Riedhammer +1
Wav2vec 2.0 (W2V2) has shown strong performance in pathological speech analysis by effectively capturing the characteristics of atypical speech. Despite its success, it remains unc…
Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features
Steffen Freisinger, Philipp Seeberger, Tobias Bocklet +1
Spoken content, such as online videos and podcasts, often spans multiple topics, which makes automatic topic segmentation essential for user navigation and downstream applications.…
Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation
Steffen Freisinger, Philipp Seeberger, Thomas Ranzenberger +2
Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hi…
Shared Multi-modal Embedding Space for Face-Voice Association
Christopher Simic, Korbinian Riedhammer, Tobias Bocklet
The FAME 2026 challenge comprises two demanding tasks: training face-voice associations combined with a multilingual setting that includes testing on languages on which the model w…
On the Difficulty of Token-Level Modeling of Dysfluency and Fluency Shaping Artifacts
Kashaf Gulzar, Dominik Wagner, Sebastian P. Bayerl +3
Automatic transcription of stuttered speech remains a challenge, even for modern end-to-end (E2E) automatic speech recognition (ASR) frameworks. Dysfluencies and fluency-shaping ar…
Generalizing to Unseen Disaster Events: A Causal View
Philipp Seeberger, Steffen Freisinger, Tobias Bocklet +1
Due to the rapid growth of social media platforms, these tools have become essential for monitoring information during ongoing disaster events. However, extracting valuable insight…
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Dong Bok Lee, Seanie Lee, Sangwoo Park +12
The reliability of large language models (LLMs) during test-time scaling is often assessed with \emph{external verifiers} or \emph{reward models} that distinguish correct reasoning…
Pitfalls and Limits in Automatic Dementia Assessment
Franziska Braun, Christopher Witzl, Andreas Erzigkeit +4
Current work on speech-based dementia assessment focuses on either feature extraction to predict assessment scales, or on the automation of existing test procedures. Most research…
Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
Dominik Wagner, Ilja Baumann, Natalie Engert +4
In this work, we present our submission to the Speech Accessibility Project challenge for dysarthric speech recognition. We integrate parameter-efficient fine-tuning with latent au…
FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
Seanie Lee, Sangwoo Park, Dong Bok Lee +5
Low-Rank Adaptation (LoRA), which introduces a product of two trainable low-rank matrices into frozen pre-trained weights, is widely used for efficient fine-tuning of language mode…
Adapter-Based Multi-Agent AVSR Extension for Pre-Trained ASR Models
Christopher Simic, Korbinian Riedhammer, Tobias Bocklet
We present an approach to Audio-Visual Speech Recognition that builds on a pre-trained Whisper model. To infuse visual information into this audio-only model, we extend it with an…
SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models
Seanie Lee, Dong Bok Lee, Dominik Wagner +5
Deploying large language models (LLMs) in real-world applications requires robust safety guard models to detect and block harmful user prompts. While large safety guard models achi…
Optimized Self-supervised Training with BEST-RQ for Speech Recognition
Ilja Baumann, Dominik Wagner, Korbinian Riedhammer +1
Self-supervised learning has been successfully used for various speech related tasks, including automatic speech recognition. BERT-based Speech pre-Training with Random-projection…
Infusing Acoustic Pause Context into Text-Based Dementia Assessment
Franziska Braun, Sebastian P. Bayerl, Florian Hönig +4
Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in…
Large Language Models for Dysfluency Detection in Stuttered Speech
Dominik Wagner, Sebastian P. Bayerl, Ilja Baumann +3
Accurately detecting dysfluencies in spoken language can help to improve the performance of automatic speech and language processing components and support the development of more…
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models
Dominik Wagner, Ilja Baumann, Korbinian Riedhammer +1
This paper explores the improvement of post-training quantization (PTQ) after knowledge distillation in the Whisper speech foundation model family. We address the challenge of outl…
Optimized Speculative Sampling for GPU Hardware Accelerators
Dominik Wagner, Seanie Lee, Ilja Baumann +3
In this work, we optimize speculative sampling for parallel hardware accelerators to improve sampling speed. We notice that substantial portions of the intermediate matrices necess…
Self-Supervised Adaptive AV Fusion Module for Pre-Trained ASR Models
Christopher Simic, Tobias Bocklet
Automatic speech recognition (ASR) has reached a level of accuracy in recent years, that even outperforms humans in transcribing speech to text. Nevertheless, all current ASR appro…
Information Type Classification with Contrastive Task-Specialized Sentence Encoders
Philipp Seeberger, Tobias Bocklet, Korbinian Riedhammer
User-generated information content has become an important information source in crisis situations. However, classification models suffer from noise and event-related biases which…
Classifying Dementia in the Presence of Depression: A Cross-Corpus Study
Franziska Braun, Sebastian P. Bayerl, Paula A. Pérez-Toro +6
Automated dementia screening enables early detection and intervention, reducing costs to healthcare systems and increasing quality of life for those affected. Depression has shared…