3 papers
cs.CL2025
Integral Transformer: Denoising Attention, Not Too Much Not Too Little
Ivan Kobyzev, Abbas Ghaddar, Dingtao Hu +1
Softmax self-attention often assigns disproportionate weight to semantically uninformative tokens such as special tokens and punctuation, a phenomenon known as attention noise. Whi…
cs.IR2025
Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval
Ruofan Hu, Yan Xia, Minjie Hong +5
Multimodal large language models (MLLMs) have seen substantial progress in recent years. However, their ability to represent multimodal information in the acoustic domain remains u…
cs.CL2024
A Novel LLM-based Two-stage Summarization Approach for Long Dialogues
Yuan-Jhe Yin, Bo-Yu Chen, Berlin Chen
Long document summarization poses a significant challenge in natural language processing due to input lengths that exceed the capacity of most state-of-the-art pre-trained language…