31 citations · 125 across the 15 of their papers we have counts for
15 papers
Native Multi-Dimensional Subquadratic Operators via Input Dependent Long Convolutions
David R. Wessels, Farhad Ramezanghorbani, Alireza Moradzadeh +9
Subquadratic alternatives to attention require compromises when applied to multi-dimensional data: standard convolutions lack global receptive fields and input dependency, while re…
Fast, Expressive SE Equivariant Networks through Weight-Sharing in Position-Orientation Space
Erik J Bekkers, Sharvaree Vadgama, Rob D Hesselink +2
Based on the theory of homogeneous spaces we derive geometrically optimal edge attributes to be used within the flexible message-passing framework. We formalize the notion of weigh…
Learned Gridification for Efficient Point Cloud Processing
Putri A. van der Linden, David W. Romero, Erik J. Bekkers
Neural operations that rely on neighborhood information are much more expensive when deployed on point clouds than on grid data due to the irregular distances between points in a p…
DNArch: Learning Convolutional Neural Architectures by Backpropagation
David W. Romero, Neil Zeghidour
We present Differentiable Neural Architectures (DNArch), a method that jointly learns the weights and the architecture of Convolutional Neural Networks (CNNs) by backpropagation. I…
Modelling Long Range Dependencies in D: From Task-Specific to a General Purpose CNN
David M. Knigge, David W. Romero, Albert Gu +5
Performant Convolutional Neural Network (CNN) architectures must be tailored to specific tasks in order to consider the length, resolution, and dimensionality of the input data. In…
Towards a General Purpose CNN for Long Range Dependencies in D
David W. Romero, David M. Knigge, Albert Gu +4
The use of Convolutional Neural Networks (CNNs) is widespread in Deep Learning due to a range of desirable model properties which result in an efficient and effective machine learn…
Relaxing Equivariance Constraints with Non-stationary Continuous Filters
Tycho F. A. van der Ouderaa, David W. Romero, Mark van der Wilk
Equivariances provide useful inductive biases in neural network modeling, with the translation equivariance of convolutional neural networks being a canonical example. Equivariance…
Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups
David M. Knigge, David W. Romero, Erik J. Bekkers
Group convolutional neural networks (G-CNNs) have been shown to increase parameter efficiency and model accuracy by incorporating geometric inductive biases. In this work, we inves…
Learning Partial Equivariances from Data
David W. Romero, Suhas Lohit
Group Convolutional Neural Networks (G-CNNs) constrain learned features to respect the symmetries in the selected group, and lead to better generalization when these symmetries app…
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes
David W. Romero, Robert-Jan Bruintjes, Jakub M. Tomczak +3
When designing Convolutional Neural Networks (CNNs), one must select the size\break of the convolutional kernels before training. Recent works show CNNs benefit from different kern…
CKConv: Continuous Kernel Convolution For Sequential Data
David W. Romero, Anna Kuzina, Erik J. Bekkers +2
Conventional neural architectures for sequential data present important limitations. Recurrent networks suffer from exploding and vanishing gradients, small effective memory horizo…
Group Equivariant Stand-Alone Self-Attention For Vision
David W. Romero, Jean-Baptiste Cordonnier
We provide a general self-attention formulation to impose group equivariance to arbitrary symmetry groups. This is achieved by defining positional encodings that are invariant to t…
Wavelet Networks: Scale-Translation Equivariant Learning From Raw Time-Series
David W. Romero, Erik J. Bekkers, Jakub M. Tomczak +1
Leveraging the symmetries inherent to specific data domains for the construction of equivariant neural networks has lead to remarkable improvements in terms of data efficiency and…
Attentive Group Equivariant Convolutional Networks
David W. Romero, Erik J. Bekkers, Jakub M. Tomczak +1
Although group convolutional networks are able to learn powerful representations based on symmetry patterns, they lack explicit means to learn meaningful relationships among them (…
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
David W. Romero, Mark Hoogendoorn
Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even…