5 papers
SALSA: Speech Aware LLM Adaptation via Learned Steering Activation Vectors
Yekaterina Yegorova, Argyrios Gerogiannis, Haolong Zheng +3
Speech-aware large language models often generalize poorly to out-of-domain settings. We propose SALSA (Speech-Aware LLM Adaptation via Learned Steering Activations), a lightweight…
LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training
Argyrios Gerogiannis, Yekaterina Yegorova, Mark Hasegawa-Johnson +1
State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to eve…
TICL+: A Case Study On Speech In-Context Learning for Children's Speech Recognition
Haolong Zheng, Yekaterina Yegorova, Mark Hasegawa-Johnson
Children's speech recognition remains challenging due to substantial acoustic and linguistic variability, limited labeled data, and significant differences from adult speech. Speec…
That's Deprecated! Understanding, Detecting, and Steering Knowledge Conflicts in Language Models for Code Generation
Jaesung Bae, Cameron Churchwell, Mitchell Hermon +5
This paper investigates how large language models (LLMs) behave when faced with discrepancies between their parametric knowledge and conflicting information contained in a prompt.…
TICL: Text-Embedding KNN For Speech In-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Models
Haolong Zheng, Yekaterina Yegorova, Mark Hasegawa-Johnson
Speech foundation models have recently demonstrated the ability to perform Speech In-Context Learning (SICL). Selecting effective in-context examples is crucial for SICL performanc…