11 citations · 11 across the 4 of their papers we have counts for
3 papers · 1 filter
Pitfalls of Evaluating Language Models with Open Benchmarks
Md. Najib Hasan, Md Mahadi Hassan Sibat, Mohammad Fakhruddin Babar +3
Open Large Language Model (LLM) benchmarks, such as HELM and BIG-Bench, provide standardized and transparent evaluation protocols that support comparative analysis, reproducibility…
LLMs as On-demand Customizable Service
Souvika Sarkar, Mohammad Fakhruddin Babar, Monowar Hasan +1
Large Language Models (LLMs) have demonstrated remarkable language understanding and generation capabilities. However, training, deploying, and accessing these models pose notable…
Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?
Souvika Sarkar, Mohammad Fakhruddin Babar, Md Mahadi Hassan +2
This paper presents a performance study of transformer language models under different hardware configurations and accuracy requirements and derives empirical observations about th…