8 citations · 19 across the 25 of their papers we have counts for
12 papers · 1 filter
PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers
Ngoc Phan Phuoc Loc, Toan Huynh La Viet, Thanh Tran Khanh +8
The rapid growth in submissions to machine learning venues has strained the scientific peer-review system and intensified interest in LLM-based automated peer reviewers. However, h…
CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era
Kaiwen Shi, Weixiang Sun, Zheyuan Zhang +3
Scientific research relies on citation integrity, yet large language models (LLMs) have introduced a critical risk: fabricated references that appear plausible but correspond to no…
ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructions
Yue Huang, Zhengzhe Jiang, Xiaonan Luo +12
Empowering large language models (LLMs) with chemical intelligence remains a challenge due to the scarcity of high-quality, domain-specific instruction-response datasets and the mi…
LLMs4All: A Review of Large Language Models Across Academic Disciplines
Yanfang Ye, Zheyuan Zhang, Tianyi Ma +26
Cutting-edge Artificial Intelligence (AI) techniques keep reshaping our view of the world. For example, Large Language Models (LLMs) based applications such as ChatGPT have shown t…
Breaking Language Barriers: Equitable Performance in Multilingual Language Models
Tanay Nagar, Grigorii Khvatskii, Anna Sokol +1
Cutting-edge LLMs have emerged as powerful tools for multilingual communication and understanding. However, LLMs perform worse in Common Sense Reasoning (CSR) tasks when prompted i…
Do Multimodal Large Language Models Understand Welding?
Grigorii Khvatskii, Yong Suk Lee, Corey Angst +3
This paper examines the performance of Multimodal LLMs (MLLMs) in skilled production work, with a focus on welding. Using a novel data set of real-world and online weld images, ann…
Dynamic Noise Preference Optimization: Self-Improvement of Large Language Models with Self-Synthetic Data
Haoyan Yang, Khiem Le, Ting Hua +7
Although LLMs have achieved significant success, their reliance on large volumes of human-annotated data has limited their potential for further scaling. In this situation, utilizi…
NGQA: A Nutritional Graph Question Answering Benchmark for Personalized Health-aware Nutritional Reasoning
Zheyuan Zhang, Yiyang Li, Nhi Ha Lan Le +9
Diet plays a critical role in human health, yet tailoring dietary reasoning to individual health conditions remains a major challenge. Nutrition Question Answering (QA) has emerged…
Class-Aware Contrastive Optimization for Imbalanced Text Classification
Grigorii Khvatskii, Nuno Moniz, Khoa Doan +1
The unique characteristics of text data make classification tasks a complex problem. Advances in unsupervised and semi-supervised learning and autoencoder architectures addressed s…
Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
Jiayi Ye, Yanbo Wang, Yue Huang +9
LLM-as-a-Judge has been widely utilized as an evaluation method in various benchmarks and served as supervised rewards in model training. However, despite their excellence in many…
LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs
Yujun Zhou, Jingdong Yang, Yue Huang +12
Artificial Intelligence (AI) is revolutionizing scientific research, yet its growing integration into laboratory environments presents critical safety challenges. Large language mo…
BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks
Anna Sokol, Elizabeth Daly, Michael Hind +4
Large language models (LLMs) are powerful tools capable of handling diverse tasks. Comparing and selecting appropriate LLMs for specific tasks requires systematic evaluation method…