4 papers
Content-Based Addressing for Long Context
Mahesh Godavarti
Rotary position embedding (RoPE) uses each token's integer position to determine the rotation applied inside attention. This works well for local token order, but increasing contex…
Journey Operators for Structured Multi-Axis Composition
Mahesh Godavarti
Many kinds of data have structure along one or more axes: words in a sentence, pixels in an image, nodes in a tree, frames in audio, or cells in a 3D volume. Along one axis, order…
Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting
Mahesh Godavarti
State-space sequence models are attractive for streaming speech because they maintain compact recurrent state, but scan-style training kernels can have unfavorable constants for sh…
Diversity vs Degrees of Freedom in Gaussian Fading Channels
Mahesh Godavarti
The classical definitions extract degrees of freedom (DOF) via $C(\snr)/\log\snr$ and diversity (DIV) via $-\log P_e(\snr)/\log\snr$, using $\log\snr$ as the common gauge for both.…