30 citations · 31 across the 10 of their papers we have counts for
6 papers · 1 filter
Structured Initialization for Vision Transformers
Jianqiao Zheng, Xueqian Li, Hemanth Saratchandran +1
Convolutional Neural Networks (CNNs) inherently encode strong inductive biases, enabling effective generalization on small-scale datasets. In this paper, we propose integrating thi…
Structured Initialization for Attention in Vision Transformers
Jianqiao Zheng, Xueqian Li, Simon Lucey
The training of vision transformer (ViT) networks on small-scale datasets poses a significant challenge. By contrast, convolutional neural networks (CNNs) have an architectural ind…
Convolutional Initialization for Data-Efficient Vision Transformers
Jianqiao Zheng, Xueqian Li, Simon Lucey
Training vision transformer networks on small datasets poses challenges. In contrast, convolutional neural networks (CNNs) can achieve state-of-the-art performance by leveraging th…
Robust Point Cloud Processing through Positional Embedding
Jianqiao Zheng, Xueqian Li, Sameera Ramasinghe +1
End-to-end trained per-point embeddings are an essential ingredient of any state-of-the-art 3D point cloud processing such as detection or alignment. Methods like PointNet, or the…
Fast Neural Scene Flow
Xueqian Li, Jianqiao Zheng, Francesco Ferroni +2
Neural Scene Flow Prior (NSFP) is of significant interest to the vision community due to its inherent robustness to out-of-distribution (OOD) effects and its ability to deal with d…
Trading Positional Complexity vs. Deepness in Coordinate Networks
Jianqiao Zheng, Sameera Ramasinghe, Xueqian Li +1
It is well noted that coordinate-based MLPs benefit -- in terms of preserving high-frequency information -- through the encoding of coordinate positions as an array of Fourier feat…