15 citations · 20 across the 18 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CL2023★ 2 cited
Exponentially Faster Language Modelling
Peter Belcak, Roger Wattenhofer
Language models only really need to use an exponential fraction of their neurons for individual inferences. As proof, we present UltraFastBERT, a BERT variant that uses 0.3% of its…
cs.LG2023★ 1 cited
Fast Feedforward Networks
Peter Belcak, Roger Wattenhofer
We break the linear link between the layer size and its inference cost by introducing the fast feedforward (FFF) architecture, a log-time alternative to feedforward networks. We de…
cs.CL2023
Examining the Emergence of Deductive Reasoning in Generative Language Models
Peter Belcak, Luca A. Lanzendörfer, Roger Wattenhofer
We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided. We observe notable differences in the performance o…