2 citations · 2 across the 6 of their papers we have counts for
7 papers
MathDuels: A Self-Play Benchmark That Grows
Zhiqiu Xu, Shibo Jin, Shreya Arya +1
As frontier language models attain near-ceiling performance on static mathematical benchmarks, existing evaluations are increasingly unable to differentiate model capabilities, lar…
Delta Activations: A Representation for Finetuned Large Language Models
Zhiqiu Xu, Amish Sethi, Mayur Naik +1
The success of powerful open source Large Language Models (LLMs) has enabled the community to create a vast collection of post-trained models adapted to specific tasks and domains.…
Generative Modeling of Weights: Generalization or Memorization?
Boya Zeng, Yida Yin, Zhiqiu Xu +1
Generative models have recently been explored for synthesizing neural network weights. These approaches take neural network checkpoints as training data and aim to generate high-pe…
Idiosyncrasies in Large Language Models
Mingjie Sun, Yida Yin, Zhiqiu Xu +2
In this work, we unveil and study idiosyncrasies in Large Language Models (LLMs) -- unique patterns in their outputs that can be used to distinguish the models. To do so, we consid…
Initializing Models with Larger Ones
Zhiqiu Xu, Yanjie Chen, Kirill Vishniakov +5
Weight initialization plays an important role in neural network training. Widely used initialization methods are proposed and evaluated for networks that are trained from scratch.…
A Coefficient Makes SVRG Effective
Yida Yin, Zhiqiu Xu, Zhiyuan Li +2
Stochastic Variance Reduced Gradient (SVRG), introduced by Johnson & Zhang (2013), is a theoretically compelling optimization method. However, as Defazio & Bottou (2019) highlight,…