2 citations · 4 across the 7 of their papers we have counts for
4 papers · 1 filter
Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation?
Tim Hartill, Joshua Bensemann, Michael Witbrock +1
A distinction is often drawn between a model's ability to predict a label for an evaluation sample that is directly memorised from highly similar training samples versus an ability…
Challenges in Annotating Datasets to Quantify Bias in Under-represented Society
Vithya Yogarajan, Gillian Dobbie, Timothy Pistotti +2
Recent advances in artificial intelligence, including the development of highly sophisticated large language models (LLM), have proven beneficial in many real-world applications. H…
Neuromodulation Gated Transformer
Kobe Knowles, Joshua Bensemann, Diana Benavides-Prado +4
We introduce a novel architecture, the Neuromodulation Gated Transformer (NGT), which is a simple implementation of neuromodulation in transformers via a multiplicative effect. We…
Input-length-shortening and text generation via attention values
Neşet Özkan Tan, Alex Yuxuan Peng, Joshua Bensemann +4
Identifying words that impact a task's performance more than others is a challenge in natural language processing. Transformers models have recently addressed this issue by incorpo…