2 citations · 3 across the 13 of their papers we have counts for
13 papers
TNT: Improving Chunkwise Training for Test-Time Memorization
Zeman Li, Ali Behrouz, Yuan Deng +5
Recurrent neural networks (RNNs) with deep test-time memorization modules, such as Titans and TTT, represent a promising, linearly-scaling paradigm distinct from Transformers. Whil…
ATLAS: Learning to Optimally Memorize the Context at Test Time
Ali Behrouz, Zeman Li, Praneeth Kacham +5
Transformers have been established as the most popular backbones in sequence modeling, mainly due to their effectiveness in in-context retrieval tasks and the ability to learn at s…
PolarQuant: Quantizing KV Caches with Polar Transformation
Insu Han, Praneeth Kacham, Amin Karbasi +2
Large language models (LLMs) require significant memory to store Key-Value (KV) embeddings in their KV cache, especially when handling long-range contexts. Quantization of these KV…
High-Dimensional Geometric Streaming for Nearly Low Rank Data
Hossein Esfandiari, Vahab Mirrokni, Praneeth Kacham +2
We study streaming algorithms for the subspace approximation problem. Given points as an insertion-only stream and a rank parameter , the su…
Optimal Communication for Classic Functions in the Coordinator Model and Beyond
Hossein Esfandiari, Praneeth Kacham, Vahab Mirrokni +2
In the coordinator model of communication with servers, given an arbitrary non-negative function , we study the problem of approximating the sum up…
Lower Bounds on Adaptive Sensing for Matrix Recovery
Praneeth Kacham, David P Woodruff
We study lower bounds on adaptive sensing algorithms for recovering low rank matrices using linear measurements. Given an matrix , a general linear measurement $S(A…
PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels
Praneeth Kacham, Vahab Mirrokni, Peilin Zhong
The quadratic time and memory complexity inherent to self-attention mechanisms, with respect to sequence length, presents a critical computational bottleneck in the training and de…
Pseudorandom Hashing for Space-bounded Computation with Applications in Streaming
Praneeth Kacham, Rasmus Pagh, Mikkel Thorup +1
We revisit Nisan's classical pseudorandom generator (PRG) for space-bounded computation (STOC 1990) and its applications in streaming algorithms. We describe a new generator, HashP…
Sub-quadratic Algorithms for Kernel Matrices via Kernel Density Estimation
Ainesh Bakshi, Piotr Indyk, Praneeth Kacham +2
Kernel matrices, as well as weighted graphs represented by them, are ubiquitous objects in machine learning, statistics and other related fields. The main drawback of using kernel…
Sketching Algorithms and Lower Bounds for Ridge Regression
Praneeth Kacham, David P. Woodruff
We give a sketching-based iterative algorithm that computes a approximate solution for the ridge regression problem where $A \in R…
Near-Optimal Algorithms for Linear Algebra in the Current Matrix Multiplication Time
Nadiia Chepurko, Kenneth L. Clarkson, Praneeth Kacham +1
In the numerical linear algebra community, it was suggested that to obtain nearly optimal bounds for various problems such as rank computation, finding a maximal linearly independe…
Reduced-Rank Regression with Operator Norm Error
Praneeth Kacham, David P. Woodruff
A common data analysis task is the reduced-rank regression problem: where and $B \in \mathbb{R}^{n \times d…
Dimensionality Reduction for Sum-of-Distances Metric
Zhili Feng, Praneeth Kacham, David P. Woodruff
We give a dimensionality reduction procedure to approximate the sum of distances of a given set of points in to any "shape" that lies in a -dimensional subspace. Here,…