85 citations · 119 across the 9 of their papers we have counts for
9 papers
Shape Mutating Expert Compression:LorExperts and BTExperts
Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3
Mixture-of-Experts (MoE) language models deliver high capacity at low per-token compute, but deploying them cheaply requires compressing their many expert weight matrices. Expert p…
KV-CAR: KV Cache Compression using Autoencoders and KV Reuse in Large Language Models
Sourjya Roy, Shrihari Sridharan, Surya Selvam +1
As Large Language Models (LLMs) scale in size and context length, the memory requirements of the key value (KV) cache have emerged as a major bottleneck during autoregressive decod…
Experts are all you need: A Composable Framework for Large Language Model Inference
Shrihari Sridharan, Sourjya Roy, Anand Raghunathan +1
Large Language Models (LLMs) have achieved state-of-the-art accuracies in a variety of natural language processing (NLP) tasks. However, this success comes at the cost of increased…
Evaluation of STT-MRAM as a Scratchpad for Training in ML Accelerators
Sourjya Roy, Cheng Wang, Anand Raghunathan
Progress in artificial intelligence and machine learning over the past decade has been driven by the ability to train larger deep neural networks (DNNs), leading to a compute deman…
PIM-DRAM: Accelerating Machine Learning Workloads using Processing in Commodity DRAM
Sourjya Roy, Mustafa Ali, Anand Raghunathan
Deep Neural Networks (DNNs) have transformed the field of machine learning and are widely deployed in many applications involving image, video, speech and natural language processi…
Pruning Filters while Training for Efficiently Optimizing Deep Learning Networks
Sourjya Roy, Priyadarshini Panda, Gopalakrishnan Srinivasan +1
Modern deep networks have millions to billions of parameters, which leads to high memory and energy requirements during training as well as during inference on resource-constrained…
TxSim:Modeling Training of Deep Neural Networks on Resistive Crossbar Systems
Sourjya Roy, Shrihari Sridharan, Shubham Jain +1
Resistive crossbars have attracted significant interest in the design of Deep Neural Network (DNN) accelerators due to their ability to natively execute massively parallel vector-m…
Gradual Channel Pruning while Training using Feature Relevance Scores for Convolutional Neural Networks
Sai Aparna Aketi, Sourjya Roy, Anand Raghunathan +1
The enormous inference cost of deep neural networks can be scaled down by network compression. Pruning is one of the predominant approaches used for deep network compression. Howev…
Proposal for a Leaky-Integrate-Fire Spiking Neuron based on Magneto-Electric Switching of Ferro-magnets
Akhilesh Jaiswal, Sourjya Roy, Gopalakrishnan Srinivasan +1
The efficiency of the human brain in performing classification tasks has attracted considerable research interest in brain-inspired neuromorphic computing. Hardware implementations…