4 papers
Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
Andrew Kiruluta, Eric Lundy, Priscilla Burity
Existing sequence to sequence models for structured language tasks rely heavily on the dot product self attention mechanism, which incurs quadratic complexity in both computation a…
Hierarchical Attention Diffusion Networks with Object Priors for Video Change Detection
Andrew Kiruluta, Eric Lundy, Andreas Lemos
We present a unified change detection pipeline that combines instance level masking, multi\-scale attention within a denoising diffusion model, and per pixel semantic classificatio…
Beyond Self Attention: A Subquadratic Fourier Wavelet Transformer with Multi Modal Fusion
Andrew Kiruluta, Andreas Lemos, Eric Lundy
We revisit the use of spectral techniques to replaces the attention mechanism in Transformers through Fourier Transform based token mixing, and present a comprehensive and novel re…
FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
Andrew Kiruluta, Eric Lundy, Andreas Lemos
We present FourierNAT, a novel non-autoregressive Transformer (NAT) architecture that employs Fourier-based mixing in the decoder to generate output sequences in parallel. While tr…