1.8k citations · 3.3k across the 25 of their papers we have counts for
5 papers · 1 filter
BDH-CQ: In-Context Learning with Recurrent Latent Reasoning
Björn Engdahl, Adrian Kosowski, Jan Chorowski +6
We introduce BDH-CQ, a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model's recurre…
The Dragon Hatchling: The Missing Link between the Transformer and Models of the Brain
Adrian Kosowski, Przemysław Uznański, Jan Chorowski +2
The relationship between computing systems and the brain has served as motivation for pioneering theoreticians since John von Neumann and Alan Turing. Uniform, scale-free biologica…
Regularizing Neural Networks by Penalizing Confident Output Distributions
Gabriel Pereyra, George Tucker, Jan Chorowski +2
We systematically explore regularizing neural networks by penalizing low entropy output distributions. We show that penalizing low entropy output distributions, which has been show…
Towards better decoding and language model integration in sequence to sequence models
Jan Chorowski, Navdeep Jaitly
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a…
End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho +1
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurre…