52 citations · 59 across the 10 of their papers we have counts for
10 papers
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
James O' Neill, Fergal Reid
Looped, weight-tied Transformers reduce parameters by reusing a single block, but decoding still stores a separate K/V cache for every recurrence step. We show that this loop-index…
Low-Rank Key Value Attention
James O'Neill, Robert Clancy, Mariia Matskevichus +1
The key-value (KV) cache is a primary memory bottleneck in Transformers. We propose Low-Rank Key-Value (LRKV) attention, which reduces KV cache memory by exploiting redundancy acro…
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models
James O' Neill, Sourav Dutta
We investigate the effects of post-training quantization and quantization-aware training on the generalization of Transformer language models. We present a new method called self-d…
I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews
James O'Neill, Polina Rozenshtein, Ryuichi Kiryo +2
Counterfactual statements describe events that did not or cannot take place. We consider the problem of counterfactual detection (CFD) in product reviews. For this purpose, we anno…
An Overview of Neural Network Compression
James O' Neill
Overparameterized networks trained to convergence have shown impressive performance in domains such as computer vision and natural language processing. Pushing state of the art on…
Compressing Deep Neural Networks via Layer Fusion
James O' Neill, Greg Ver Steeg, Aram Galstyan
This paper proposes \textit{layer fusion} - a model compression technique that discovers which weights to combine and then fuses weights of similar fully-connected, convolutional a…
Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answering
Angrosh Mandya, James O'Neill, Danushka Bollegala +1
The Conversational Question Answering (CoQA) task involves answering a sequence of inter-related conversational questions about a contextual paragraph. Although existing approaches…
Transfer Reward Learning for Policy Gradient-Based Text Generation
James O' Neill, Danushka Bollegala
Task-specific scores are often used to optimize for and evaluate the performance of conditional text generation systems. However, such scores are non-differentiable and cannot be u…
Error-Correcting Neural Sequence Prediction
James O' Neill, Danushka Bollegala
We propose a novel neural sequence prediction method based on \textit{error-correcting output codes} that avoids exact softmax normalization and allows for a tradeoff between speed…
Curriculum-Based Neighborhood Sampling For Sequence Prediction
James O' Neill, Danushka Bollegala
The task of multi-step ahead prediction in language models is challenging considering the discrepancy between training and testing. At test time, a language model is required to ma…