34 citations · 42 across the 16 of their papers we have counts for
6 papers · 1 filter
On the Design Space Between Transformers and Recursive Neural Nets
Jishnu Ray Chowdhury, Cornelia Caragea
In this paper, we study two classes of models, Recursive Neural Networks (RvNNs) and Transformers, and show that a tight connection between them emerges from the recent development…
Investigating Recurrent Transformers with Dynamic Halt
Jishnu Ray Chowdhury, Cornelia Caragea
In this paper, we comprehensively study the inductive biases of two major approaches to augmenting Transformers with a recurrent mechanism: (1) the approach of incorporating a dept…
Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability
Jishnu Ray Chowdhury, Cornelia Caragea
Binary Balanced Tree RvNNs (BBT-RvNNs) enforce sequence composition according to a preset balanced binary tree structure. Thus, their non-linear recursion depth is just …
Efficient Beam Tree Recursion
Jishnu Ray Chowdhury, Cornelia Caragea
Beam Tree Recursive Neural Network (BT-RvNN) was recently proposed as a simple extension of Gumbel Tree RvNN and it was shown to achieve state-of-the-art length generalization perf…
Beam Tree Recursive Cells
Jishnu Ray Chowdhury, Cornelia Caragea
We propose Beam Tree Recursive Cell (BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We…
Monotonic Location Attention for Length Generalization
Jishnu Ray Chowdhury, Cornelia Caragea
We explore different ways to utilize position-based cross-attention in seq2seq networks to enable length generalization in algorithmic tasks. We show that a simple approach of inte…