30 citations · 31 across the 10 of their papers we have counts for
11 papers
The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
Jianqiao Zheng, Hemanth Saratchandran, Simon Lucey
Implicit Neural Representations (INRs) have revolutionized continuous signal modeling, yet they struggle to recover fine-grained details within finite training budgets. While empir…
From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
Jianqiao Zheng, Cameron Gordon, Yiping Ji +2
Task-agnostic tabular foundation models such as TabPFN have achieved impressive performance on tabular learning tasks, yet the origins of their inductive biases remain poorly under…
Cutting the Skip: Training Residual-Free Transformers
Yiping Ji, James Martens, Jianqiao Zheng +5
Transformers have achieved remarkable success across a wide range of applications, a feat often attributed to their scalability. Yet training them without skip (residual) connectio…
Structured Initialization for Vision Transformers
Jianqiao Zheng, Xueqian Li, Hemanth Saratchandran +1
Convolutional Neural Networks (CNNs) inherently encode strong inductive biases, enabling effective generalization on small-scale datasets. In this paper, we propose integrating thi…
Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
Hemanth Saratchandran, Jianqiao Zheng, Yiping Ji +2
This paper questions whether the strong performance of softmax attention in transformers stems from producing a probability distribution over inputs. Instead, we argue that softmax…
Structured Initialization for Attention in Vision Transformers
Jianqiao Zheng, Xueqian Li, Simon Lucey
The training of vision transformer (ViT) networks on small-scale datasets poses a significant challenge. By contrast, convolutional neural networks (CNNs) have an architectural ind…
Convolutional Initialization for Data-Efficient Vision Transformers
Jianqiao Zheng, Xueqian Li, Simon Lucey
Training vision transformer networks on small datasets poses challenges. In contrast, convolutional neural networks (CNNs) can achieve state-of-the-art performance by leveraging th…
Robust Point Cloud Processing through Positional Embedding
Jianqiao Zheng, Xueqian Li, Sameera Ramasinghe +1
End-to-end trained per-point embeddings are an essential ingredient of any state-of-the-art 3D point cloud processing such as detection or alignment. Methods like PointNet, or the…
Fast Neural Scene Flow
Xueqian Li, Jianqiao Zheng, Francesco Ferroni +2
Neural Scene Flow Prior (NSFP) is of significant interest to the vision community due to its inherent robustness to out-of-distribution (OOD) effects and its ability to deal with d…
Trading Positional Complexity vs. Deepness in Coordinate Networks
Jianqiao Zheng, Sameera Ramasinghe, Xueqian Li +1
It is well noted that coordinate-based MLPs benefit -- in terms of preserving high-frequency information -- through the encoding of coordinate positions as an array of Fourier feat…
Rethinking Positional Encoding
Jianqiao Zheng, Sameera Ramasinghe, Simon Lucey
It is well noted that coordinate based MLPs benefit -- in terms of preserving high-frequency information -- through the encoding of coordinate positions as an array of Fourier feat…