From the 1 of 5 linked papers with an AI index.
5 papers
Mask-Based Priors Are More Persistent than Query-Key Initializations
Mingze Ma, Hemanth Saratchandran, Cameron Gordon +1
Transformers do not merely lack data on some Boolean extrapolation tasks; they generalize in a systematically wrong way. Recent work on generalization on the unseen has shown that,…
Memory Efficient Tabular Foundation Models
Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon +2
The paper studies how to reduce the memory footprint of tabular foundation models like TabPFN using compression techniques, achieving up to 7.6× memory savings with little performa…
From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
Jianqiao Zheng, Cameron Gordon, Yiping Ji +2
Task-agnostic tabular foundation models such as TabPFN have achieved impressive performance on tabular learning tasks, yet the origins of their inductive biases remain poorly under…
SineLoRA: Sine-Activated Delta Compression
Cameron Gordon, Yiping Ji, Hemanth Saratchandran +2
Resource-constrained weight deployment is a task of immense practical importance. Recently, there has been interest in the specific task of \textit{Delta Compression}, where partie…
Efficient Learning With Sine-Activated Low-rank Matrices
Yiping Ji, Hemanth Saratchandran, Cameron Gordon +2
Low-rank decomposition has emerged as a vital tool for enhancing parameter efficiency in neural network architectures, gaining traction across diverse applications in machine learn…