258 citations · 935 across the 92 of their papers we have counts for
93 papers
Toward AI-Resilient Assessment in Computer Science Courses in an AI-Native World
Anshumali Shrivastava
AI-native course assessments in senior computer science courses and related fields should grade students by \emph{AI-resilient skill}: the ability to achieve outcomes beyond a stro…
RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
Sahil Joshi, Agniva Chowdhury, Amar Kanakamedala +3
Softmax Attention has a quadratic time complexity in sequence length, which becomes prohibitive to run at long contexts, even with highly optimized GPU kernels. For example, FlashA…
Adaptive Sampling for Deep Learning via Efficient Nonparametric Proxies
Shabnam Daghaghi, Benjamin Coleman, Benito Geordie +1
Data sampling is an effective method to improve the training speed of neural networks, with recent results demonstrating that it can even break the neural scaling laws. These resul…
Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces
Aditya Desai, Benjamin Meisburger, Zichang Liu +1
Recommendation systems (RS) for items (e.g., movies, books) and ads are widely used to tailor content to users on various internet platforms. Traditionally, recommendation models a…
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Zichang Liu, Jue Wang, Tri Dao +8
Large language models (LLMs) with hundreds of billions of parameters have sparked a new wave of exciting AI applications. However, they are computationally expensive at inference t…
In defense of parameter sharing for model-compression
Aditya Desai, Anshumali Shrivastava
When considering a model architecture, there are several ways to reduce its memory footprint. Historically, popular approaches included selecting smaller architectures and creating…