3 papers
cs.AI2026
RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following
Tianjun Pan, Xuan Lin, Wenyan Yang +7
Rubric-based evaluation has become a prevailing paradigm for evaluating instruction following in large language models (LLMs). Despite its widespread use, the reliability of these…
cs.SE2025
LUK: Empowering Log Understanding with Expert Knowledge from Large Language Models
Lipeng Ma, Weidong Yang, Sihang Jiang +6
Logs play a critical role in providing essential information for system monitoring and troubleshooting. Recently, with the success of pre-trained language models (PLMs) and large l…
cs.SE2025
AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model
Lipeng Ma, Weidong Yang, Yixuan Li +6
Automated log analysis is crucial to ensure high availability and reliability of complex systems. The advent of LLMs in NLP has ushered in a new era of language model-driven automa…