7 citations · 11 across the 11 of their papers we have counts for
12 papers
The Hidden Ratio in Adam: Stable Structure, Compression, and Sign Dynamics
Yihe Zhou, Tongtian Zhu, Yingxiao Huo +4
Adam is the default optimizer for training modern deep neural networks, yet its adaptive behavior remains poorly understood due to the complex interaction between its first- and se…
Distributed Team Orchestration via Supervisor Networks: Convergence, Optimality, and Resilience
Juntian Zhu, Guanpu Chen, Tongtian Zhu +3
In this paper, we study zero-sum potential team games with a supervisor network, where agents rely on supervisor-provided belief information rather than accurate common beliefs. Th…
Mobius Learning: Cyclic Depth Folding in Transformers
Tongtian Zhu
Transformer-based language models organize computation along an ordered depth axis, where shallow and deep blocks often develop distinct representational roles. We challenge the co…
DICE: Data Influence Cascade in Decentralized Learning
Tongtian Zhu, Wenhao Li, Can Wang +1
Decentralized learning offers a promising approach to crowdsource data consumptions and computational workloads across geographically distributed compute interconnected through pee…
On the Surprising Effectiveness of a Single Global Merging in Decentralized Learning
Tongtian Zhu, Tianyu Zhang, Mingze Wang +2
Decentralized learning provides a scalable alternative to parameter-server-based training, yet its performance is often hindered by limited peer-to-peer communication. In this pape…
Lie Symmetry Net: Preserving Conservation Laws in Modelling Financial Market Dynamics via Differential Equations
Xuelian Jiang, Tongtian Zhu, Yingxiang Xu +3
This paper employs a novel Lie symmetries-based framework to model the intrinsic symmetries within financial market. Specifically, we introduce Lie symmetry net (LSN), which charac…
Adversarial Erasing with Pruned Elements: Towards Better Graph Lottery Ticket
Yuwen Wang, Shunyu Liu, Kaixuan Chen +5
Graph Lottery Ticket (GLT), a combination of core subgraph and sparse subnetwork, has been proposed to mitigate the computational cost of deep Graph Neural Networks (GNNs) on large…
Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization
Kaixuan Chen, Shunyu Liu, Tongtian Zhu +5
Graph Neural Networks (GNNs) have emerged as a powerful category of learning architecture for handling graph-structured data. However, existing GNNs typically ignore crucial struct…
Lookaround Optimizer: steps around, 1 step average
Jiangtao Zhang, Shunyu Liu, Jie Song +3
Weight Average (WA) is an active research topic due to its simplicity in ensembling deep networks and the effectiveness in promoting generalization. Existing weight average approac…
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
Tongtian Zhu, Fengxiang He, Kaixuan Chen +2
Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on massive devices simultaneously without the control of a central server. However, existing theorie…
Contrastive Identity-Aware Learning for Multi-Agent Value Decomposition
Shunyu Liu, Yihe Zhou, Jie Song +5
Value Decomposition (VD) aims to deduce the contributions of agents for decentralized policies in the presence of only global rewards, and has recently emerged as a powerful credit…
Topology-aware Generalization of Decentralized SGD
Tongtian Zhu, Fengxiang He, Lan Zhang +3
This paper studies the algorithmic stability and generalizability of decentralized stochastic gradient descent (D-SGD). We prove that the consensus model learned by D-SGD is $\math…