From the 2 of 5 linked papers with an AI index.
6 citations · 8 across the 3 of their papers we have counts for
5 papers
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
Sangjin Kim, Yuseon Choi, Jungjun Oh +2
LightRot introduces a lightweight rotation scheme and a dedicated hardware accelerator that enable energy‑efficient, low‑bit inference for large language models such as LLaMA2‑13B…
GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
Sangjin Kim, Yuseon Choi, Byeongcheol Kim +2
GyRot introduces a co-designed quantization framework and hardware accelerator that combine coarse rotation with fine-grained group quantization, enabling accurate 4-bit inference…
ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Serving
Yuseon Choi, Jingu Lee, Jungjun Oh +5
Mixture-of-Experts (MoE) models have become the dominant architecture for large-scale language models, yet on-premises serving remains fundamentally memory-bound as batching turns…
SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
Yuseon Choi, Sangjin Kim, Jungjun Oh +3
MoE models offer efficient scaling through conditional computation, but their large parameter size and expensive expert offloading make on-device deployment challenging. Existing a…
SeVeDo: A Heterogeneous Transformer Accelerator for Low-Bit Inference via Hierarchical Group Quantization and SVD-Guided Mixed Precision
Yuseon Choi, Sangjin Kim, Jungjun Oh +2
Low-bit quantization is a promising technique for efficient transformer inference by reducing computational and memory overhead. However, aggressive bitwidth reduction remains chal…