5 papers
Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization
Patrik Czakó, Gábor Kertész, Sándor Szénási
Post-training quantization (PTQ) is one of the most practical ways to reduce the serving cost of Large Language Models (LLMs), but activation quantization remains difficult because…
Ensembles of Large Language Models for Identifying EQ-5D Studies in PubMed Based on Their Abstracts
Zhyar Rzgar K. Rostam, Márta Péntek, János Tibor Czere +3
The rapid increase in scientific publications leads to the fact that manual study screening in systematic literature reviews (SLRs) is increasingly resource consuming, inefficient,…
EQ-5D Classification Using Biomedical Entity-Enriched Pre-trained Language Models and Multiple Instance Learning
Zhyar Rzgar K Rostam, Gábor Kertész
The EQ-5D (EuroQol 5-Dimensions) is a standardized instrument for the evaluation of health-related quality of life. In health economics, systematic literature reviews (SLRs) depend…
SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs
Patrik Czakó, Gábor Kertész, Sándor Szénási
We present SmoothRot, a novel post-training quantization technique to enhance the efficiency of 4-bit quantization in Large Language Models (LLMs). SmoothRot addresses the critical…
Turning LLM Activations Quantization-Friendly
Patrik Czakó, Gábor Kertész, Sándor Szénási
Quantization effectively reduces the serving costs of Large Language Models (LLMs) by speeding up data movement through compressed parameters and enabling faster operations via int…