Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Learning High-Frequency Functions Made Easy with Sinusoidal Positional Encoding
Chuanhao Sun, Zhihang Yuan, Kai Xu +4
Fourier features based positional encoding (PE) is commonly used in machine learning tasks that involve learning high-frequency features from low-dimensional inputs, such as 3D vie…
cs.CL2024
Banyan: Improved Representation Learning with Explicit Structure
Mattia Opper, N. Siddharth
We present Banyan, a model that efficiently learns semantic representations by leveraging explicit hierarchical structure. While transformers excel at scale, they struggle in low-r…