16 citations · 48 across the 16 of their papers we have counts for
5 papers · 1 filter
A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
Davis Wertheimer, Haochen Shen, Ahan Gupta +6
The large KV-cache size of modern LLMs creates a barrier to efficient deployment. Recent work has explored replacing attention layers' RoPE positional embeddings with alternative d…
Enhancing Training Efficiency Using Packing with Flash Attention
Achintya Kundu, Rhui Dih Lee, Laura Wynter +2
Padding is often used in tuning LLM models by adding special tokens to shorter training examples to match the length of the longest sequence in each batch. While this ensures unifo…
SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach
Tianshi Wang, Jinyang Li, Ruijie Wang +7
This paper introduces SudokuSens, a generative framework for automated generation of training data in machine-learning-based Internet-of-Things (IoT) applications, such that the ge…
AI Foundation Models for Weather and Climate: Applications, Design, and Implementation
S. Karthik Mukkavilli, Daniel Salles Civitarese, Johannes Schmude +12
Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been incr…
neuralRank: Searching and ranking ANN-based model repositories
Nirmit Desai, Linsong Chu, Raghu K. Ganti +2
Widespread applications of deep learning have led to a plethora of pre-trained neural network models for common tasks. Such models are often adapted from other models via transfer…