11 citations · 23 across the 5 of their papers we have counts for
6 papers
Pitfalls of Evaluating Language Models with Open Benchmarks
Md. Najib Hasan, Md Mahadi Hassan Sibat, Mohammad Fakhruddin Babar +3
Open Large Language Model (LLM) benchmarks, such as HELM and BIG-Bench, provide standardized and transparent evaluation protocols that support comparative analysis, reproducibility…
Large Language Models for IT Automation Tasks: Are We There Yet?
Md Mahadi Hassan, John Salvador, Akond Rahman +1
LLMs show promise in code generation, yet their effectiveness for IT automation tasks, particularly for tools like Ansible, remains understudied. Existing benchmarks rely primarily…
Introducing "Forecast Utterance" for Conversational Data Science
Md Mahadi Hassan, Alex Knipper, Shubhra Kanti Karmaker
Envision an intelligent agent capable of assisting users in conducting forecasting tasks through intuitive, natural conversations, without requiring in-depth knowledge of the under…
ChatGPT as your Personal Data Scientist
Md Mahadi Hassan, Alex Knipper, Shubhra Kanti Karmaker Santu
The rise of big data has amplified the need for efficient, user-friendly automated machine learning (AutoML) tools. However, the intricacy of understanding domain-specific data and…
Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?
Souvika Sarkar, Mohammad Fakhruddin Babar, Md Mahadi Hassan +2
This paper presents a performance study of transformer language models under different hardware configurations and accuracy requirements and derives empirical observations about th…
AutoML to Date and Beyond: Challenges and Opportunities
Shubhra Kanti Karmaker Santu, Md. Mahadi Hassan, Micah J. Smith +3
As big data becomes ubiquitous across domains, and more and more stakeholders aspire to make the most of their data, demand for machine learning tools has spurred researchers to ex…