5 papers
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Suho Yoo, Hyunjong Ok, Jongmin Choi +2
As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-onl…
FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping
Jongmin Choi, Joon Son Chung
Spiking Neural Networks (SNNs) are a promising framework for event-driven temporal processing. Prior work has improved temporal modeling through richer neuron dynamics and network-…
LAMB: LLM-based Audio Captioning with Modality Gap Bridging via Cauchy-Schwarz Divergence
Hyeongkeun Lee, Jongmin Choi, KiHyun Nam +1
Automated Audio Captioning aims to describe the semantic content of input audio. Recent works have employed large language models (LLMs) as a text decoder to leverage their reasoni…
Diffusion-Link: Diffusion Probabilistic Model for Bridging the Audio-Text Modality Gap
KiHyun Nam, Jongmin Choi, Hyeongkeun Lee +2
Contrastive audio-language pretraining yields powerful joint representations, yet a persistent audio-text modality gap limits the benefits of coupling multimodal encoders with larg…
Fork-Merge Decoding: Enhancing Multimodal Understanding in Audio-Visual Large Language Models
Chaeyoung Jung, Youngjoon Jang, Jongmin Choi +1
The goal of this work is to enhance balanced multimodal understanding in audio-visual large language models (AV-LLMs) by addressing modality bias without additional training. In cu…