Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Integral Transformer: Denoising Attention, Not Too Much Not Too Little
Ivan Kobyzev, Abbas Ghaddar, Dingtao Hu +1
Softmax self-attention often assigns disproportionate weight to semantically uninformative tokens such as special tokens and punctuation, a phenomenon known as attention noise. Whi…
cs.CL2024
A Novel LLM-based Two-stage Summarization Approach for Long Dialogues
Yuan-Jhe Yin, Bo-Yu Chen, Berlin Chen
Long document summarization poses a significant challenge in natural language processing due to input lengths that exceed the capacity of most state-of-the-art pre-trained language…