16 citations · 25 across the 8 of their papers we have counts for
4 papers · 1 filter
Gradient Sparsification For Masked Fine-Tuning of Transformers
James O' Neill, Sourav Dutta
Fine-tuning pretrained self-supervised language models is widely adopted for transfer learning to downstream tasks. Fine-tuning can be achieved by freezing gradients of the pretrai…
Aligned Weight Regularizers for Pruning Pretrained Neural Networks
James O' Neill, Sourav Dutta, Haytham Assem
While various avenues of research have been explored for iterative pruning, little is known what effect pruning has on zero-shot test performance and its potential implications on…
-Neighbor Based Curriculum Sampling for Sequence Prediction
James O' Neill, Danushka Bollegala
Multi-step ahead prediction in language models is challenging due to the discrepancy between training and test time processes. At test time, a sequence predictor is required to mak…
Meta-Embedding as Auxiliary Task Regularization
James O' Neill, Danushka Bollegala
Word embeddings have been shown to benefit from ensambling several word embedding sources, often carried out using straightforward mathematical operations over the set of word vect…