10 citations · 38 across the 19 of their papers we have counts for
22 papers
A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
Davis Wertheimer, Haochen Shen, Ahan Gupta +6
The large KV-cache size of modern LLMs creates a barrier to efficient deployment. Recent work has explored replacing attention layers' RoPE positional embeddings with alternative d…
xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding
Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim +4
Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multiple-token drafting in speculative…
Using Span Queries to Optimize for Cache and Attention Locality
Paul Castro, Nick Mitchell, Nathan Ordonez +3
Clients are evolving beyond chat completion, and now include a variety of innovative inference-time scaling and deep reasoning techniques. At the same time, inference servers remai…
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
Krish Agarwal, Rishi Astra, Adnan Hoque +4
We present HadaCore, a modified Fast Walsh-Hadamard Transform (FWHT) algorithm optimized for the Tensor Cores present in modern GPU hardware. HadaCore follows the recursive structu…
Transforming the Hybrid Cloud for Emerging AI Workloads
Deming Chen, Alaa Youssef, Ruchi Pendse +42
This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet th…
CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation
Claudius Krause, Michele Faucci Giannelli, Gregor Kasieczka +66
We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of i…
Accelerating Production LLMs with Combined Token/Embedding Speculators
Davis Wertheimer, Joshua Rosenkranz, Thomas Parnell +4
This technical report describes the design and training of novel speculative decoding draft models, for accelerating the inference speeds of large language models in a production e…
Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition
Adnan Hoque, Less Wright, Chih-Chieh Yang +2
We propose an implementation of an efficient fused matrix multiplication kernel for W4A16 quantized inference, where we perform dequantization and GEMM in a fused kernel using a Sp…
SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach
Tianshi Wang, Jinyang Li, Ruijie Wang +7
This paper introduces SudokuSens, a generative framework for automated generation of training data in machine-learning-based Internet-of-Things (IoT) applications, such that the ge…
TP-Aware Dequantization
Adnan Hoque, Mudhakar Srivatsa, Chih-Chieh Yang +1
In this paper, we present a novel method that reduces model inference latency during distributed deployment of Large Language Models (LLMs). Our contribution is an optimized infere…
Rethinking Data-driven Networking with Foundation Models: Challenges and Opportunities
Franck Le, Mudhakar Srivatsa, Raghu Ganti +1
Foundational models have caused a paradigm shift in the way artificial intelligence (AI) systems are built. They have had a major impact in natural language processing (NLP), and s…
State Action Separable Reinforcement Learning
Ziyao Zhang, Liang Ma, Kin K. Leung +2
Reinforcement Learning (RL) based methods have seen their paramount successes in solving serial decision-making and control problems in recent years. For conventional RL formulatio…
Neural Network Tomography
Liang Ma, Ziyao Zhang, Mudhakar Srivatsa
Network tomography, a classic research problem in the realm of network monitoring, refers to the methodology of inferring unmeasured network attributes using selected end-to-end pa…
SENSE: Semantically Enhanced Node Sequence Embedding
Swati Rallapalli, Liang Ma, Mudhakar Srivatsa +4
Effectively capturing graph node sequences in the form of vector embeddings is critical to many applications. We achieve this by (i) first learning vector embeddings of single grap…
neuralRank: Searching and ranking ANN-based model repositories
Nirmit Desai, Linsong Chu, Raghu K. Ganti +2
Widespread applications of deep learning have led to a plethora of pre-trained neural network models for common tasks. Such models are often adapted from other models via transfer…
Actor Conditioned Attention Maps for Video Action Detection
Oytun Ulutan, Swati Rallapalli, Mudhakar Srivatsa +2
While observing complex events with multiple actors, humans do not assess each actor separately, but infer from the context. The surrounding context provides essential information…
Object Localization and Size Estimation from RGB-D Images
ShreeRanjani SrirangamSridharan, Oytun Ulutan, Shehzad Noor Taus Priyo +2
Depth sensing cameras (e.g., Kinect sensor, Tango phone) can acquire color and depth images that are registered to a common viewpoint. This opens the possibility of developing algo…
Beyond Spatial Auto-Regressive Models: Predicting Housing Prices with Satellite Imagery
Archith J. Bency, Swati Rallapalli, Raghu K. Ganti +2
When modeling geo-spatial data, it is critical to capture spatial correlations for achieving high accuracy. Spatial Auto-Regression (SAR) is a common tool used to model such data,…
Prediction-based Online Trajectory Compression
Arlei Silva, Ramya Raghavendra, Mudhakar Srivatsa +1
Recent spatio-temporal data applications, such as car-shar\-ing and smart cities, impose new challenges regarding the scalability and timeliness of data processing systems. Traject…
Joint Source Selection and Data Extrapolation in Social Sensing for Disaster Response
Mohammad Hosseini, Nooreddin Nagibolhosseini, Amotz Barnoy +14
This paper complements the large body of social sensing literature by developing means for augmenting sensing data with inference results that "fill-in" missing pieces. It specific…
Picking vs. Guessing Secrets: A Game-Theoretic Analysis (Technical Report)
MHR Khouzani, Piotr Mardziel, Carlos Cid +1
Choosing a hard-to-guess secret is a prerequisite in many security applications. Whether it is a password for user authentication or a secret key for a cryptographic primitive, pic…
Quantifying Information Leakage in Finite Order Deterministic Programs
Ji Zhu, Mudhakar Srivatsa
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information…