4 citations · 6 across the 7 of their papers we have counts for
4 papers · 1 filter
Delta Activations: A Representation for Finetuned Large Language Models
Zhiqiu Xu, Amish Sethi, Mayur Naik +1
The success of powerful open source Large Language Models (LLMs) has enabled the community to create a vast collection of post-trained models adapted to specific tasks and domains.…
Generative Modeling of Weights: Generalization or Memorization?
Boya Zeng, Yida Yin, Zhiqiu Xu +1
Generative models have recently been explored for synthesizing neural network weights. These approaches take neural network checkpoints as training data and aim to generate high-pe…
Initializing Models with Larger Ones
Zhiqiu Xu, Yanjie Chen, Kirill Vishniakov +5
Weight initialization plays an important role in neural network training. Widely used initialization methods are proposed and evaluated for networks that are trained from scratch.…
A Coefficient Makes SVRG Effective
Yida Yin, Zhiqiu Xu, Zhiyuan Li +2
Stochastic Variance Reduced Gradient (SVRG), introduced by Johnson & Zhang (2013), is a theoretically compelling optimization method. However, as Defazio & Bottou (2019) highlight,…