4 papers · 1 filter
When Verifiable Counts Depend on Wording: Auditing Wording Robustness in Instruction Following
Qishi Zhan, Seoyeon Jang, Zihan Dong +3
Verifiable instruction-following benchmarks often express each constraint through one fixed template. We test whether scores remain stable when the operational requirement is uncha…
Disentangling Language Roles in Multilingual LLM Task Execution
Qishi Zhan, Minxuan Hu, Seoyeon Jang +7
Multilingual LLMs are increasingly used when instruction, source content, and required response languages do not coincide. Existing benchmarks have expanded multilingual instructio…
Assessment of Generative Named Entity Recognition in the Era of Large Language Models
Qi Zhan, Yile Wang, Hui Huang
Named entity recognition (NER) is evolving from a sequence labeling task into a generative paradigm with the rise of large language models (LLMs). We conduct a systematic evaluatio…
How Order-Sensitive Are LLMs? OrderProbe for Deterministic Structural Reconstruction
Zhaolu Kang, Yingjie He, Kehan Jiang +18
Large language models (LLMs) excel at semantic understanding, yet their ability to reconstruct internal structure from scrambled inputs remains underexplored. Sentence-level restor…