4 citations · 6 across the 4 of their papers we have counts for
4 papers
Causal Graph in Language Model Rediscovers Cortical Hierarchy in Human Narrative Processing
Zhengqi He, Taro Toyoizumi
Understanding how humans process natural language has long been a vital research direction. The field of natural language processing (NLP) has recently experienced a surge in the d…
On the Overlooked Structure of Stochastic Gradients
Zeke Xie, Qian-Yuan Tang, Mingming Sun +1
Stochastic gradients closely relate to both optimization and generalization of deep neural networks (DNNs). Some works attempted to explain the success of stochastic optimization f…
Spontaneous Emerging Preference in Two-tower Language Model
Zhengqi He, Taro Toyoizumi
The ever-growing size of the foundation language model has brought significant performance gains in various types of downstream tasks. With the existence of side-effects brought ab…
Sparse Double Descent: Where Network Pruning Aggravates Overfitting
Zheng He, Zeke Xie, Quanzhi Zhu +1
People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity. However, our work…