5 papers
Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Models
Yuchen Su, Shaoxin Zhong, Yonghua Zhu +6
Puns represent a typical linguistic phenomenon that exploits polysemy and phonetic ambiguity to generate humour, posing unique challenges for natural language understanding. Within…
FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation
Siyang He, Qiqi Wang, Xiaoran Liu +8
Despite the non-autoregressive potential of diffusion language models (dLLMs), existing decoding strategies demonstrate positional bias, failing to fully unlock the potential of ar…
DiRL: An Efficient Post-Training Framework for Diffusion Language Models
Ying Zhu, Jiaxin Wan, Xiaoran Liu +7
Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accele…
Thus Spake Long-Context Large Language Model
Xiaoran Liu, Ruixiao Li, Mianqiu Huang +11
Long context is an important topic in Natural Language Processing (NLP), running through the development of NLP architectures, and offers immense opportunities for Large Language M…
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
Xiaoran Liu, Siyang He, Qiqi Wang +9
Large Language Models struggle with memory demands from the growing Key-Value (KV) cache as context lengths increase. Existing compression methods homogenize head dimensions or rel…