16 citations · 48 across the 16 of their papers we have counts for
17 papers
A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
Davis Wertheimer, Haochen Shen, Ahan Gupta +6
The large KV-cache size of modern LLMs creates a barrier to efficient deployment. Recent work has explored replacing attention layers' RoPE positional embeddings with alternative d…
xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding
Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim +4
Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multiple-token drafting in speculative…
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
Krish Agarwal, Rishi Astra, Adnan Hoque +4
We present HadaCore, a modified Fast Walsh-Hadamard Transform (FWHT) algorithm optimized for the Tensor Cores present in modern GPU hardware. HadaCore follows the recursive structu…
Next generation Co-Packaged Optics Technology to Train & Run Generative AI Models in Data Centers and Other Computing Applications
John Knickerbocker, Jean Benoit Heroux, Griselda Bonilla +15
We report on the successful design and fabrication of optical modules using a 50 micron pitch polymer waveguide interface, integrated for low loss, high density optical data transf…
Transforming the Hybrid Cloud for Emerging AI Workloads
Deming Chen, Alaa Youssef, Ruchi Pendse +42
This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet th…
Flexible and Effective Mixing of Large Language Models into a Mixture of Domain Experts
Rhui Dih Lee, Laura Wynter, Raghu Kiran Ganti
We present a toolkit for creating low-cost Mixture-of-Domain-Experts (MOE) from trained models. The toolkit can be used for creating a mixture from models or from adapters. We perf…
Enhancing Training Efficiency Using Packing with Flash Attention
Achintya Kundu, Rhui Dih Lee, Laura Wynter +2
Padding is often used in tuning LLM models by adding special tokens to shorter training examples to match the length of the longest sequence in each batch. While this ensures unifo…
Accelerating Production LLMs with Combined Token/Embedding Speculators
Davis Wertheimer, Joshua Rosenkranz, Thomas Parnell +4
This technical report describes the design and training of novel speculative decoding draft models, for accelerating the inference speeds of large language models in a production e…
Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition
Adnan Hoque, Less Wright, Chih-Chieh Yang +2
We propose an implementation of an efficient fused matrix multiplication kernel for W4A16 quantized inference, where we perform dequantization and GEMM in a fused kernel using a Sp…
SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach
Tianshi Wang, Jinyang Li, Ruijie Wang +7
This paper introduces SudokuSens, a generative framework for automated generation of training data in machine-learning-based Internet-of-Things (IoT) applications, such that the ge…
TP-Aware Dequantization
Adnan Hoque, Mudhakar Srivatsa, Chih-Chieh Yang +1
In this paper, we present a novel method that reduces model inference latency during distributed deployment of Large Language Models (LLMs). Our contribution is an optimized infere…
Foundation Models for Generalist Geospatial Artificial Intelligence
Johannes Jakubik, Sujit Roy, C. E. Phillips +30
Significant progress in the development of highly adaptable and reusable Artificial Intelligence (AI) models is expected to have a significant impact on Earth science and remote se…
AI Foundation Models for Weather and Climate: Applications, Design, and Implementation
S. Karthik Mukkavilli, Daniel Salles Civitarese, Johannes Schmude +12
Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been incr…
Rethinking Data-driven Networking with Foundation Models: Challenges and Opportunities
Franck Le, Mudhakar Srivatsa, Raghu Ganti +1
Foundational models have caused a paradigm shift in the way artificial intelligence (AI) systems are built. They have had a major impact in natural language processing (NLP), and s…
neuralRank: Searching and ranking ANN-based model repositories
Nirmit Desai, Linsong Chu, Raghu K. Ganti +2
Widespread applications of deep learning have led to a plethora of pre-trained neural network models for common tasks. Such models are often adapted from other models via transfer…
Beyond Spatial Auto-Regressive Models: Predicting Housing Prices with Satellite Imagery
Archith J. Bency, Swati Rallapalli, Raghu K. Ganti +2
When modeling geo-spatial data, it is critical to capture spatial correlations for achieving high accuracy. Spatial Auto-Regression (SAR) is a common tool used to model such data,…
Joint Source Selection and Data Extrapolation in Social Sensing for Disaster Response
Mohammad Hosseini, Nooreddin Nagibolhosseini, Amotz Barnoy +14
This paper complements the large body of social sensing literature by developing means for augmenting sensing data with inference results that "fill-in" missing pieces. It specific…