position embeddings 1random fourier features 1sequential recommendation 1temporal modeling 1transformers 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.LG2026
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Yiwen Chen, Joshua Ainslie, Krzysztof Choromanski +4
The paper introduces ClockRoPE, a method that uses random Fourier rotations to create periodic position embeddings for transformers, improving temporal routine modeling in sequenti…
cs.LG2024
Linear Transformer Topological Masking with Graph Random Features
Isaac Reid, Kumar Avinava Dubey, Deepali Jain +12
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance. Topological masking, a type of relativ…