6 papers · 1 filter
Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective
Chen Zhang, Yuxin Cheng, Chenchen Ding +5
Zeroth-order (ZO) optimization enables memory-efficient training of neural networks by estimating gradients via forward passes only, eliminating the need for backpropagation. Howev…
NTK-Guided Implicit Neural Teaching
Chen Zhang, Wei Zuo, Bingyang Cheng +4
Implicit Neural Representations (INRs) parameterize continuous signals via multilayer perceptrons (MLPs), enabling compact, resolution-independent modeling for tasks like image, au…
Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
Chen Zhang, Weixin Bu, Wendong Xu +3
Transformers have achieved remarkable success in time series modeling, yet their internal mechanisms remain opaque. This work demystifies the Transformer encoder by establishing it…
Nonparametric Teaching for Graph Property Learners
Chen Zhang, Weixin Bu, Zeyi Ren +3
Inferring properties of graph-structured data, e.g., the solubility of molecules, essentially involves learning the implicit mapping from graphs to their properties. This learning…
ASMR: Activation-sharing Multi-resolution Coordinate Networks For Efficient Inference
Jason Chun Lok Li, Steven Tin Sui Luo, Le Xu +1
Coordinate network or implicit neural representation (INR) is a fast-emerging method for encoding natural signals (such as images and videos) with the benefits of a compact neural…
Nonparametric Teaching of Implicit Neural Representations
Chen Zhang, Steven Tin Sui Luo, Jason Chun Lok Li +2
We investigate the learning of implicit neural representation (INR) using an overparameterized multilayer perceptron (MLP) via a novel nonparametric teaching perspective. The latte…