4 papers
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Liang-Hsuan Tseng, Yi-Chang Chen, Kuan-Yi Lee +2
Recent efforts target spoken language models (SLMs) that not only listen but also speak for more natural human-LLM interaction. Joint speech-text modeling is a promising direction…
Adaptive vector steering: A training-free, layer-wise intervention for hallucination mitigation in large audio and multimodal models
Tsung-En Lin, Kuan-Yi Lee, Hung-Yi Lee
Large Audio-Language Models and Multi-Modal Large Language Models have demonstrated strong capabilities in tasks such as Audio Question Answering (AQA), Audio Captioning, and Autom…
Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning
Kuan-Yi Lee, Tsung-En Lin, Hung-Yi Lee
Recent advancements in large multimodal models (LMMs) have shown strong capabilities in audio understanding. However, most systems rely solely on end-to-end reasoning, limiting int…
AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schema
Ting-Chun Liu, Ching-Yu Hsu, Kuan-Yi Lee +2
Prompt injection attacks pose a significant challenge to the safe deployment of Large Language Models (LLMs) in real-world applications. While prompt-based detection offers a light…