most citedImproving Transformers with Dynamically Composable Multi-Head Attention

1 citations · 1 across the 6 of their papers we have counts for

collaborators
Showing cs.LGShow all

5 papers · 1 filter

cs.LG2025

FlowCast-ODE: Continuous Hourly Weather Forecasting with Dynamic Flow Matching and ODE Solver

Shuangshuang He, Yuanting Zhang, Hongli Liang +3

Data-driven hourly weather forecasting models often face the challenge of error accumulation in long-term predictions. The problem is exacerbated by non-physical temporal discontin…

cs.LG2025

PCDCNet: A Surrogate Model for Air Quality Forecasting with Physical-Chemical Dynamics and Constraints

Shuo Wang, Yun Cheng, Qingye Meng +6

Air quality forecasting (AQF) is critical for public health and environmental management, yet remains challenging due to the complex interplay of emissions, meteorology, and chemic…

cs.LG2025

CNCast: Leveraging 3D Swin Transformer and DiT for Enhanced Regional Weather Forecasting

Hongli Liang, Yuanting Zhang, Qingye Meng +2

This study introduces a cutting-edge regional weather forecasting model based on the SwinTransformer 3D architecture. This model is specifically designed to deliver precise hourly…

cs.LG2025

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections

Da Xiao, Qingye Meng, Shengping Li +1

We propose MUltiway Dynamic Dense (MUDD) connections, a simple yet effective method to address the limitations of residual connections and enhance cross-layer information flow in T…

cs.LG2024★ 1 cited

Improving Transformers with Dynamically Composable Multi-Head Attention

Da Xiao, Qingye Meng, Shengping Li +1

Multi-Head Attention (MHA) is a key component of Transformer. In MHA, attention heads work independently, causing problems such as low-rank bottleneck of attention score matrices a…