23 citations · 43 across the 26 of their papers we have counts for
29 papers
The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models
Ying He, Sihang Jiang, Xingzhou Chen +6
Cultural taboo safety is essential for deploying large language models (LLMs), as culturally insensitive outputs may cause offense or even social harm. However, existing cultural b…
From Isolated Tasks to Structured Capabilities: A Multilayer Taxonomy for Large Language Models
Shixin Fang, Jiachen Wo, Wenjuan Qin +2
Large language model (LLM) evaluation spans diverse tasks and benchmarks, yet evidence remains organized around tasks rather than the capabilities they probe. This fragmentation li…
When RAG Meets Query Planning: Logical Query Trees for Resolving Exploratory Reasoning Problems
Ganlin Xu, Linghao Zhang, Zhitao Yin +7
Retrieval-Augmented Generation (RAG) effectively grounds large language models (LLMs) in external knowledge but struggles with \textbf{exploratory reasoning problems (ERPs)} that a…
From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training
Zishang Jiang, Tingyun Li, Jinyi Han +7
Reinforcement learning (RL) has become a widely adopted technique for improving large language models (LLMs) on complex tasks. Despite this progress, existing RL methods still face…
ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models
Tingyun Li, Zishang Jiang, Jinyi Han +8
Large reasoning models rely on long chain-of-thought to achieve strong performance, but applying such reasoning uniformly incurs high computational cost. Existing efficiency-orient…
SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment
Sihang Jiang, Lipeng Ma, Zhonghua Hong +9
Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, failing to accumulate experience…
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
Jiaqing Liang, Jinyi Han, Weijia Li +15
Long-horizon large language model (LLM) agents are fundamentally limited by context. As interactions become longer, tool descriptions, retrieved memories, and raw environmental fee…
Do Large Language Models Truly Understand Cross-cultural Differences?
Shiwei Guo, Sihang Jiang, Qianxi He +6
In recent years, large language models (LLMs) have demonstrated strong performance on multilingual tasks. Given its wide range of applications, cross-cultural understanding capabil…
ComLQ: Benchmarking Complex Logical Queries in Information Retrieval
Ganlin Xu, Zhitao Yin, Linghao Zhang +6
Information retrieval (IR) systems play a critical role in navigating information overload across various applications. Existing IR benchmarks primarily focus on simple queries tha…
Why Did Apple Fall: Evaluating Curiosity in Large Language Models
Haoyu Wang, Sihang Jiang, Yuyan Chen +4
Curiosity serves as a pivotal conduit for human beings to discover and learn new knowledge. Recent advancements of large language models (LLMs) in natural language processing have…
Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts
Xinyi Wang, Jinyi Han, Zishang Jiang +7
Reinforcement Learning (RL) has become a key driver for enhancing the long chain-of-thought (CoT) reasoning capabilities of Large Language Models (LLMs). However, prevalent methods…
Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
Zishang Jiang, Jinyi Han, Tingyun Li +7
Reinforcement Learning with Verifiable Rewards (RLVR) has become a widely adopted technique for enhancing the reasoning ability of Large Language Models (LLMs). However, the effect…
LogReasoner: Empowering LLMs with Expert-like Coarse-to-Fine Reasoning for Automated Log Analysis
Lipeng Ma, Yixuan Li, Weidong Yang +7
Log analysis is crucial for monitoring system health and diagnosing failures in complex systems. Recent advances in large language models (LLMs) offer new opportunities for automat…
CultureScope: A Dimensional Lens for Probing Cultural Understanding in LLMs
Jinghao Zhang, Sihang Jiang, Shiwei Guo +7
As large language models (LLMs) are increasingly deployed in diverse cultural environments, evaluating their cultural understanding capability has become essential for ensuring tru…
Your Models Have Thought Enough: Training Large Reasoning Models to Stop Overthinking
Jinyi Han, Ying Huang, Ying Liao +11
Large Reasoning Models (LRMs) have achieved impressive performance on challenging tasks, yet their deep reasoning often incurs substantial computational costs. To achieve efficient…
A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models
Jinyi Han, Xinyi Wang, Haiquan Zhao +9
Recent advances in self-refinement have demonstrated significant potential for improving the outputs of large language models (LLMs) through iterative refinement. However, most exi…
AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model
Lipeng Ma, Weidong Yang, Yixuan Li +6
Automated log analysis is crucial to ensure high availability and reliability of complex systems. The advent of LLMs in NLP has ushered in a new era of language model-driven automa…
EDGE: Enhanced Grounded GUI Understanding with Enriched Multi-Granularity Synthetic Data
Xuetian Chen, Hangcheng Li, Jiaqing Liang +2
Autonomous agents operating on the graphical user interfaces (GUIs) of various applications hold immense practical value. Unlike the large language model (LLM)-based methods which…
LUK: Empowering Log Understanding with Expert Knowledge from Large Language Models
Lipeng Ma, Weidong Yang, Sihang Jiang +6
Logs play a critical role in providing essential information for system monitoring and troubleshooting. Recently, with the success of pre-trained language models (PLMs) and large l…
VCEval: Rethinking What is a Good Educational Video and How to Automatically Evaluate It
Xiaoxuan Zhu, Zhouhong Gu, Sihang Jiang +3
Online courses have significantly lowered the barrier to accessing education, yet the varying content quality of these videos poses challenges. In this work, we focus on the task o…
Enhancing Quantitative Reasoning Skills of Large Language Models through Dimension Perception
Yuncheng Huang, Qianyu He, Jiaqing Liang +3
Quantities are distinct and critical components of texts that characterize the magnitude properties of entities, providing a precise perspective for the understanding of natural la…
Piecing Together Clues: A Benchmark for Evaluating the Detective Skills of Large Language Models
Zhouhong Gu, Lin Zhang, Jiangjie Chen +8
Detectives frequently engage in information detection and reasoning simultaneously when making decisions across various cases, especially when confronted with a vast amount of info…
Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation
Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye +16
New Natural Langauge Process~(NLP) benchmarks are urgently needed to align with the rapid development of large language models (LLMs). We present Xiezhi, the most comprehensive eva…
Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release
Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye +7
Domain knowledge refers to the in-depth understanding, expertise, and familiarity with a specific subject, industry, field, or area of special interest. The existing benchmarks are…