1 citations · 1 across the 5 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Benchmarking and Understanding Compositional Relational Reasoning of LLMs
Ruikang Ni, Da Xiao, Qingye Meng +3
Compositional relational reasoning (CRR) is a hallmark of human intelligence, but we lack a clear understanding of whether and how existing transformer large language models (LLMs)…
cs.LG2024★ 1 cited
Improving Transformers with Dynamically Composable Multi-Head Attention
Da Xiao, Qingye Meng, Shengping Li +1
Multi-Head Attention (MHA) is a key component of Transformer. In MHA, attention heads work independently, causing problems such as low-rank bottleneck of attention score matrices a…