21 citations · 33 across the 8 of their papers we have counts for
7 papers
MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding
Yu Gong, Kailash Budhathoki, Taeho Kim +2
Mixture-of-Experts (MoE) layers increase model capacity without proportionally increasing arithmetic, but their sparse expert computation is difficult to execute efficiently during…
KernelSight-LM: A Kernel-Level LLM Inference Simulator
Xiteng Yao, Taeho Kim, Hengzhi Pei +7
As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to m…
LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs
Taeho Kim, Yanming Wang, Vatshank Chaturvedi +4
Fine-tuning pre-trained large language models (LLMs) with limited hardware presents challenges due to GPU memory constraints. Various distributed fine-tuning methods have been prop…
Tensor Slicing and Optimization for Multicore NPUs
Rafael Sousa, Marcio Pereira, Yongin Kwon +5
Although code generation for Convolution Neural Network (CNN) models has been extensively studied, performing efficient data slicing and parallelization for highly-constrai\-ned Mu…
Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization
Seongmin Park, Beomseok Kwon, Jieun Lim +3
Uniform-precision neural network quantization has gained popularity since it simplifies densely packed arithmetic unit for high computing capability. However, it ignores heterogene…
CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution
Taeho Kim, Yongin Kwon, Jemin Lee +1
Mobile devices run deep learning models for various purposes, such as image classification and speech recognition. Due to the resource constraints of mobile devices, researchers ha…
Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment
Jemin Lee, Misun Yu, Yongin Kwon +1
To adopt convolutional neural networks (CNN) for a range of resource-constrained targets, it is necessary to compress the CNN models by performing quantization, whereby precision r…