4 papers · 1 filter
Attention under Bounded Key-Query Similarity: Space Complexity and Transformer Anisotropy
Justin Y. Chen, Ying Feng, Piotr Indyk +3
A growing body of work on KV-cache compression aims to reduce the memory requirements of autoregressive generation. These methods compress the stored context while approximately pr…
AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance
Chenxi Wang, Ying Feng, Hongjie Fang +4
Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retarg…
Towards Tight Bounds for Streaming Attention
Justin Y. Chen, Ying Feng, Piotr Indyk +3
The attention mechanism is a cornerstone of modern transformer architectures. However, its expressive power comes at the cost of quadratic runtime and linear space usage. In partic…
Provable Quantization with Randomized Hadamard Transform
Ying Feng, Piotr Indyk, Michael Kapralov +2
Vector quantization via random projection followed by scalar quantization is a fundamental primitive in machine learning, with applications ranging from similarity search to federa…