1 citations · 1 across the 15 of their papers we have counts for
16 papers
Belief-Trajectory Energy: Measuring the Path to a Prediction
Jiahao Ying, Wei Tang, Boxian Ai +7
Large language models (LLMs) progressively revise their predictions across Transformer layers, yet we typically observe only the final output, discarding the trajectory through whi…
HarmProfile: Characterizing Harmful Distributions in Frontier LLMs
Zhouyuan Ma, Yutao Wu, Hanxun Huang +6
Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analysis. Consequently, little is kn…
OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents
Jiahao Ying, Boxian Ai, Wei Tang +2
Skills, i.e., structured workflow instructions distilled for large language models (LLMs), are becoming an increasingly important mechanism for improving agent performance on real-…
UniPPTBench: A Unified Benchmark for Presentation Generation Across Diverse Input Settings
Bo Zhao, Maosheng Pang, Chen Zhang +3
Existing works typically focus on presentation generation under isolated input settings, whereas real-world use cases span diverse scenarios, including vague user prompts, long doc…
Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
Jiahao Ying, Mingbao Lin, Qianru Sun +1
Mixture-of-Experts (MoE) architectures have emerged as a promising direction, offering efficiency and scalability by activating only a subset of parameters during inference. Howeve…
EffiEval: Efficient and Generalizable Model Evaluation via Capability Coverage Maximization
Yaoning Wang, Jiahao Ying, Yixin Cao +2
The rapid advancement of large language models (LLMs) and the development of increasingly large and diverse evaluation benchmarks have introduced substantial computational challeng…
Disentangling Language and Culture for Evaluating Multilingual Large Language Models
Jiahao Ying, Wei Tang, Yiran Zhao +3
This paper introduces a Dual Evaluation Framework to comprehensively assess the multilingual capabilities of LLMs. By decomposing the evaluation along the dimensions of linguistic…
Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metric
Yixin Cao, Jiahao Ying, Yaoning Wang +3
Large Language Models (LLMs) have become indispensable across academia, industry, and daily applications, yet current evaluation methods struggle to keep pace with their rapid deve…
Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches
Yuhang Zhou, Xutian Chen, Yixin Cao +8
Recent progress in large language models (LLMs) has outpaced the development of effective evaluation methods. Traditional benchmarks rely on task-specific metrics and static datase…
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks
Yixin Cao, Shibo Hong, Xinze Li +24
Large Language Models (LLMs) are advancing at an amazing speed and have become indispensable across academia, industry, and daily applications. To keep pace with the status quo, th…
EvoWiki: Evaluating LLMs on Evolving Knowledge
Wei Tang, Yixin Cao, Yang Deng +8
Knowledge utilization is a critical aspect of LLMs, and understanding how they adapt to evolving knowledge is essential for their effective deployment. However, existing benchmarks…
LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement
Jiahao Ying, Mingbao Lin, Yixin Cao +5
This paper introduces the innovative "LLMs-as-Instructors" framework, which leverages the advanced Large Language Models (LLMs) to autonomously enhance the training of smaller targ…
A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential
Wei Tang, Yixin Cao, Jiahao Ying +4
Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs). Targeting its bottleneck of retriever performance,…
QRMeM: Unleash the Length Limitation through Question then Reflection Memory Mechanism
Bo Wang, Heyan Huang, Yixin Cao +3
While large language models (LLMs) have made notable advancements in natural language processing, they continue to struggle with processing extensive text. Memory mechanism offers…
Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models
Jiahao Ying, Yixin Cao, Yushi Bai +7
Large language models (LLMs) have achieved impressive performance across various natural language benchmarks, prompting a continual need to curate more difficult datasets for large…
FedMKGC: Privacy-Preserving Federated Multilingual Knowledge Graph Completion
Wei Tang, Zhiqian Wu, Yixin Cao +2
Knowledge graph completion (KGC) aims to predict missing facts in knowledge graphs (KGs), which is crucial as modern KGs remain largely incomplete. While training KGC models on mul…