9 papers
The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
Jianqiao Zheng, Hemanth Saratchandran, Simon Lucey
Implicit Neural Representations (INRs) have revolutionized continuous signal modeling, yet they struggle to recover fine-grained details within finite training budgets. While empir…
Robust Physical Adversarial Patches Using Dynamically Optimized Clusters
Harrison Bagley, Will Meakin, Simon Lucey +2
Physical adversarial attacks on deep learning systems is concerning due to the ease of deploying such attacks, usually by placing an adversarial patch in a scene to manipulate the…
From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
Jianqiao Zheng, Cameron Gordon, Yiping Ji +2
Task-agnostic tabular foundation models such as TabPFN have achieved impressive performance on tabular learning tasks, yet the origins of their inductive biases remain poorly under…
Cutting the Skip: Training Residual-Free Transformers
Yiping Ji, James Martens, Jianqiao Zheng +5
Transformers have achieved remarkable success across a wide range of applications, a feat often attributed to their scalability. Yet training them without skip (residual) connectio…
Leaner Transformers: More Heads, Less Depth
Hemanth Saratchandran, Damien Teney, Simon Lucey
Transformers have reshaped machine learning by utilizing attention mechanisms to capture complex patterns in large datasets, leading to significant improvements in performance. Thi…
SineLoRA: Sine-Activated Delta Compression
Cameron Gordon, Yiping Ji, Hemanth Saratchandran +2
Resource-constrained weight deployment is a task of immense practical importance. Recently, there has been interest in the specific task of \textit{Delta Compression}, where partie…