3 papers
cs.CL2024
MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures
Jinjie Ni, Fuzhao Xue, Xiang Yue +5
Evaluating large language models (LLMs) is challenging. Traditional ground-truth-based benchmarks fail to capture the comprehensiveness and nuance of real-world queries, while LLM-…
cs.CL2023
A Survey on Semantic Processing Techniques
Rui Mao, Kai He, Xulang Zhang +4
Semantic processing is a fundamental research domain in computational linguistics. In the era of powerful pre-trained language models and large language models, the advancement of…
cs.LG2023
Finding the Pillars of Strength for Multi-Head Attention
Jinjie Ni, Rui Mao, Zonglin Yang +2
Recent studies have revealed some issues of Multi-Head Attention (MHA), e.g., redundancy and over-parameterization. Specifically, the heads of MHA were originally designed to atten…