5 papers
Intrinsic Entropy of Context Length Scaling in LLMs
Jingzhe Shi, Qinwei Ma, Hongyi Liu +3
Long Context Language Models have drawn great attention in the past few years. There has been work discussing the impact of long context on Language Model performance: some find th…
Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning
Hang Zhao, Qile P. Chen, Yijing Barry Zhang +1
Both encoder-only models (e.g., BERT, RoBERTa) and large language models (LLMs, e.g., Llama3) have been widely used for text classification tasks. However, there is a lack of syste…
When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation
Anna Min, Chenxu Hu, Yi Ren +1
Though end-to-end speech-to-text translation has been a great success, we argue that the cascaded speech-to-text translation model still has its place, which is usually criticized…
A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation
Anna Min, Chenxu Hu, Yi Ren +1
Current research in speech-to-speech translation (S2ST) primarily concentrates on translation accuracy and speech naturalness, often overlooking key elements like paralinguistic in…
MINT: Boosting Audio-Language Model via Multi-Target Pre-Training and Instruction Tuning
Hang Zhao, Yifei Xin, Zhesong Yu +3
In the realm of audio-language pre-training (ALP), the challenge of achieving cross-modal alignment is significant. Moreover, the integration of audio inputs with diverse distribut…