most citedInput-length-shortening and text generation via attention values

2 citations · 3 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2023

Neuromodulation Gated Transformer

Kobe Knowles, Joshua Bensemann, Diana Benavides-Prado +4

We introduce a novel architecture, the Neuromodulation Gated Transformer (NGT), which is a simple implementation of neuromodulation in transformers via a multiplicative effect. We…

cs.CL2023★ 2 cited

Input-length-shortening and text generation via attention values

Neşet Özkan Tan, Alex Yuxuan Peng, Joshua Bensemann +4

Identifying words that impact a task's performance more than others is a challenge in natural language processing. Transformers models have recently addressed this issue by incorpo…