2 papers
cs.LG2025
MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving
Jungi Lee, Junyong Park, Soohyun Cha +2
Reduced-precision data formats are crucial for cost-effective serving of large language models (LLMs). While numerous reduced-precision formats have been introduced thus far, they…
cs.LG2024
MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models
Taehyun Kim, Kwanseok Choi, Youngmock Cho +3
Mixture-of-Experts (MoE) large language models (LLM) have memory requirements that often exceed the GPU memory capacity, requiring costly parameter movement from secondary memories…