70 citations · 83 across the 31 of their papers we have counts for
6 papers · 1 filter
PolicyLLM: Towards Excellent Comprehension of Public Policy for Large Language Models
Han Bao, Penghao Zhang, Yue Huang +9
Large Language Models (LLMs) are increasingly integrated into real-world decision-making, including in the domain of public policy. Yet, their ability to comprehend and reason abou…
Automated Benchmark Generation from Domain Guidelines Informed by Bloom's Taxonomy
Si Chen, Le Huy Khiem, Annalisa Szymanski +3
Open-ended question answering (QA) evaluates a model's ability to perform contextualized reasoning beyond factual recall. This challenge is especially acute in practice-based domai…
Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
Peiyu Li, Xiuxiu Tang, Si Chen +4
Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale. Existing evaluat…
KEO: Knowledge Extraction on OMIn via Knowledge Graphs and RAG for Safety-Critical Aviation Maintenance
Kuangshi Ai, Jonathan A. Karr, Meng Jiang +2
We present Knowledge Extraction on OMIn (KEO), a domain-specific knowledge extraction and reasoning framework with large language models (LLMs) in safety-critical contexts. Using t…
LatentPrompt: Optimizing Promts in Latent Space
Mateusz Bystroński, Grzegorz Piotrowski, Nitesh V. Chawla +1
Recent advances have shown that optimizing prompts for Large Language Models (LLMs) can significantly improve task performance, yet many optimization techniques rely on heuristics…
SMOTExT: SMOTE meets Large Language Models
Mateusz Bystroński, Mikołaj Hołysz, Grzegorz Piotrowski +2
Data scarcity and class imbalance are persistent challenges in training robust NLP models, especially in specialized domains or low-resource settings. We propose a novel technique,…