5 citations · 17 across the 12 of their papers we have counts for
8 papers · 1 filter
Substance or Style: What Does Your Image Embedding Know?
Cyrus Rashtchian, Charles Herrmann, Chun-Sung Ferng +5
Probes are small networks that predict properties of underlying data from embeddings, and they provide a targeted, effective way to illuminate the information contained in embeddin…
Leveraging redundancy in attention with Reuse Transformers
Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit +5
Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications i…
Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation
Srinadh Bhojanapalli, Ayan Chakrabarti, Himanshu Jain +3
State-of-the-art transformer models use pairwise dot-product based self-attention, which comes at a computational cost quadratic in the input sequence length. In this paper, we inv…
Real-Time Edge Classification: Optimal Offloading under Token Bucket Constraints
Ayan Chakrabarti, Roch Guérin, Chenyang Lu +1
To deploy machine learning-based algorithms for real-time applications with strict latency constraints, we consider an edge-computing setting where a subset of inputs are offloaded…
Neural Network-Inspired Analog-to-Digital Conversion to Achieve Super-Resolution with Low-Precision RRAM Devices
Weidong Cao, Liu Ke, Ayan Chakrabarti +1
Recent works propose neural network- (NN-) inspired analog-to-digital converters (NNADCs) and demonstrate their great potentials in many emerging applications. These NNADCs often r…
Backprop with Approximate Activations for Memory-efficient Network Training
Ayan Chakrabarti, Benjamin Moseley
Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers. This presents a practical concern w…