9 papers
Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective
Chen Zhang, Yuxin Cheng, Chenchen Ding +5
Zeroth-order (ZO) optimization enables memory-efficient training of neural networks by estimating gradients via forward passes only, eliminating the need for backpropagation. Howev…
ELPO: Ensemble Learning Based Prompt Optimization for Large Language Models
Qing Zhang, Bing Xu, Xudong Zhang +9
The remarkable performance of Large Language Models (LLMs) highly relies on crafted prompts. However, manual prompt engineering is a laborious process, creating a core bottleneck f…
AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
Wendong Xu, Chujie Chen, He Xiao +8
Large Language Model (LLM) inference services demand exceptionally high availability and low latency, yet multi-GPU Tensor Parallelism (TP) makes them vulnerable to single-GPU fail…
NTK-Guided Implicit Neural Teaching
Chen Zhang, Wei Zuo, Bingyang Cheng +4
Implicit Neural Representations (INRs) parameterize continuous signals via multilayer perceptrons (MLPs), enabling compact, resolution-independent modeling for tasks like image, au…
Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
Chen Zhang, Weixin Bu, Wendong Xu +3
Transformers have achieved remarkable success in time series modeling, yet their internal mechanisms remain opaque. This work demystifies the Transformer encoder by establishing it…
Enhancing Robustness of Implicit Neural Representations Against Weight Perturbations
Wenyong Zhou, Yuxin Cheng, Zhengwu Liu +3
Implicit Neural Representations (INRs) encode discrete signals in a continuous manner using neural networks, demonstrating significant value across various multimedia applications.…
MINR: Efficient Implicit Neural Representations for Multi-Image Encoding
Wenyong Zhou, Taiqiang Wu, Zhengwu Liu +3
Implicit Neural Representations (INRs) aim to parameterize discrete signals through implicit continuous functions. However, formulating each image with a separate neural network~(t…
Nonparametric Teaching for Graph Property Learners
Chen Zhang, Weixin Bu, Zeyi Ren +3
Inferring properties of graph-structured data, e.g., the solubility of molecules, essentially involves learning the implicit mapping from graphs to their properties. This learning…
Nonparametric Teaching of Implicit Neural Representations
Chen Zhang, Steven Tin Sui Luo, Jason Chun Lok Li +2
We investigate the learning of implicit neural representation (INR) using an overparameterized multilayer perceptron (MLP) via a novel nonparametric teaching perspective. The latte…